Two different questions - is 42 too old to change careers? Not at all.( Well, I would say that - I changed to horticulture at that age!)
Loads of people come to horticulture as a profession after another career, and they have many advantages over the students who've gone from school into full-time horticultural education: common sense and maturity, realism from their experience of the working world, patience ( to deal with tetchy, changeable clients) and most of all an enthusiasm for gardening and plants. Some young people I've taught have the enthusiasm, but the rest they tend to develop quite slowly once they set out on their own!
The second question is whether physically you are going to be able to do this, once you've got the knowledge and the confidence. I think the answer to that is, really it depends on what you want to do in horticulture. Working in a nursery propagating plants, or in a garden centre displaying them, selling them and most of all giving good advice to customers - both of these are less physically demanding compared to general garden maintenance work, though anyone who's done the jobs will tell you about back pain, RSI and carpal tunnel syndrome.( And don't even THINK about the joy of Christmas in the garden centre!)
If you want to set up your own business maintaining gardens, one option would be to test out how you would handle this by working initially for one of the bigger maintenance franchises, where you wouldn't be working alone and where your knowledge of things like pruning would be paired with someone else's ability to lift the lawnmower into the back of a truck.
However I think the answer is not to think of yourself as totally isolated.
Use the opportunity for networking when you study - easiest if it's in a college, but with many of the distance learners there's an opportunity to contact other students and get to know them. Or use the Facebook group you'll find from this forum, if you are into that sort of thing; there is The Garden Network online as well.
There's nothing unusual about bringing in other people to do parts of a large job, like clearing a garden - if your speciality is plants and plant maintenance and you get to know someone reasonably local who's a good landscaper, or someone else who has chainsaw training, you will be able to sub-contract that part of the work for an agreed price, or even just an informal 'I'll find you suitable plants to go round the pond you're building for x, and come and plant them, if you bring your mattock and spend a morning on this border with me'.
Be confident in what you can do, and tell your customers what you will 'organise' instead. You don't have to do one-off labouring work, or trim high hedges, if it isn't your main business. Muscles for labouring jobs can be hired and for some work like tree maintenance you'd need a specialist with the right insurance in place anyway. Your customers will respect your knowledge and they should take your advice!
There are so many different aspects to gardening as a career; go for it!