I don't think calc-based physics is that much of a bonus for med-school bound folks -- you'll do fine on the MCAT without it, and it sounds like your GPA will be happier without it. (And, in retrospect, though I took calc and calc-based physics, I regret it and would have rather taken calc I and stats plus regular physics, for reasons of my grades, usefulness of material, and sanity.)
Of course, the flip side is that people who take calc and calc-based physics have a competitive edge if they excel in those classes. But it doesn't mean they'll be better doctors, and many schools recognize this. Depends on the competitiveness of the school is my guess.