Youth is wasted on the young.... such a cliche but true.
Get your training out of the way when you are in your 20s. Have babies and tend to your career in your 30s. Through your 40s enjoy making new friends through your kids friends (you'll have more in common with people whose kids are the ages of your kids than with your own age cohort) and enjoy sharing the world with your kids (or taking up hobbies artistic, athletic or otherwise). As you enter your 50s and 60s you've got more money, your kids are more independent, and God willing, you still have your health and stamina. I've got doc friends who at that age are enjoying vacations in exotic vacations or restful weekends at their beach houses. With clean living (no smoking, weight management, minimal booze, safe driving) you can enjoy travel, cross country skiing, sailing, and cultural events well into your 70s and 80s like some of my old doctor friends (and non-doctor friends).
You will have to work hard but physicians are well rewarded for their effort and you will have the free time to enjoy it.