Difficult? Well, it's hard work. You can work as hard as you want in medical school. But difficult, like hard to actually succeed even with determined effort? Well...I guess that depends on the individual.
If you're not sure about going to medical school, think it over carefully before making your final decision. It will likely be far more work than college; if that's not your thing, then it's probably not right for you at this time.
If you're worried about whether or not you'll be able to do it - and asking about people dropping out - it's uncommon, but not unheard of for people to drop out. A few change their minds, every once in a while someone can't keep up with the work.