I've looked into this at a lot of schools, because I want to do it. While there are schools where you can do both degrees simulataneously in 4 years, the vast majority require you to take a year off to do the MPH coursework, usually between your 2nd and 3rd years of medical school.
University of Rochester is an example of a school that is more flexible regarding the timing. At Rochester you can do your MPH year first, and then do MS1-MS4 without interruption; or you can do the MPH between 2nd and 3rd year; or you can even do it between 1st and 2nd year (which is unusual). The advantage to doing your MPH first is that you get to start and graduate with the same MD class rather than switching in the middle, and you have more time to work on your MPH thesis as well. As I've said, however, at most schools you do the MPH between MS2 and MS3.
Most schools want you to be accepted to medical school before you apply for the MPH program, and many schools (though not all) want you to wait to apply for the MPH until you've already been in medical school long enough to be a known quantity for them. Other schools allow or want you to apply as soon as you have been accepted to medical school but before the year begins, though I don't know of any schools that won't allow you to apply later, after you've begun medical school, if you decide later on that you want to get the MPH.
OHSU, Harvard and Drexel have you apply to both programs at the same time, much as MSTP students apply to the MD and PhD programs simultaneously. (There may be other schools that do this too, but these are the ones that I know.) Even so, at all three of these schools you can decide after you are already in medical school to apply for the MPH. At OHSU (and perhaps at other schools, though I don't know) one advantage of applying for the MD and the MPH programs together is that you do not have disadvantaged OOS status like OOS applicants to the MD program have.
Many schools that don't have MPH programs themselves have formal arrangements with other schools that do. UCSF has a formal arrangement with UC Berkeley, and Duke has an arrangement with UNC-Chapel Hill, for example. You leave your MD school for a year, but they facilitate everything for you and your financial aid doesn't need to be renegotiated, and the schools are so close that you don't have to move anywhere. Similarly, Einstein will let you take a year off to pursue the MPH at Columbia, but I BELIEVE that in their case that year is free of tuition. I haven't looked into the details of this program, so I can't tell you anything else about it, but it sounds like a pretty damned good deal if it really works that way.
Other schools are at least happy to let you go for a year to do the MPH, even if they don't have anything set up with another school. The people at Penn State told me that I would be more than welcome to take a year off to do the MPH at Johns Hopkins, for example, and I'm sure that they wouldn't mind if I decided to go to UNC instead.
One final consideration is that not all schools that offer the MPH have a separate School of Public Health. Some schools have a department of Community and Preventive Medicine (or something similar) through which the MPH is obtained. When you look at the rankings in US News and World Report, you have to look at both Schools of Public Health and graduate programs offered through these other departments. US News ranks them both, and Rochester's program, for example, is highly regarded; but they don't have a separate school of public health, so you might overlook it if you didn't realize that you have to look elsewhere to read about it.
Hope that helps.