It also depends what you want to do with your life. It's all relative. There are different levels of competition even within MD and PhD programs.
If you're trying to match plastic surgery, medical school is going to be a lot harder than someone who doesn't really care and will take any reasonable program in a non-competitive specialty. Heck, if you're trying to match top tier rad onc, it's probably better that you not only do very well in med school, but you spend a year or more out of med school doing solid research as well.
If you're trying to become a tenure track professor at a major research institution, graduate school is going to be a lot harder than someone who is there screwing around because they couldn't figure out what they wanted to do with their life otherwise.