Yes, I was another one of those who DREADED med school. I seriously thought about deferring to do some soul-searching. But, you know, it turns out that med school was actually one of the greatest decisions I've made in my life.
Try not to let yourself get too freaked out about the workload. You can handle it. The admissions people have been doing this for a long time and they know what it takes to make it through med school. If you weren't capable of doing the work, they wouldn't have let you in.
Sometimes the negative comments on this site can give you a skewed perception of what it's really like. Yes, med school is stressful and time-consuming at times. Now that I have the pace and intensity of med school as a basis for comparison, the pace of my undergrad Psychology classes seems like a total joke to me. However, it's also easier to feel motivated to work harder when it's something directly relevant to your interests, and I've learned a lot of cool and important things in just this first year.
I was worried about having a bleak existence of four years of constant studying. Turns out that I do have a decent amount of free time now that I have figured out how to study efficiently (once you've taken a few tests, you develop a sense for what the high yield topics are and whether it's worth your time to attend class in person or not). As long as you are willing to make a reasonable effort to keep up with the work, are willing to seek out tutoring if you find you're struggling, etc. then you should have no trouble staying afloat.
I guess I tend to think that if you're doing it for the right reasons (because you have a genuine interest and passion for medicine), then you'll probably be happy you took the plunge even though certain times may be difficult. The road isn't really that long when you break it up into small chunks. Don't focus on making it through the whole four years. Focus on just making it through the first test, then the next, then the next...then the whole semester will be over before you know it.
The first year really flew by for me. It really wasn't that bad, overall. I can't think of any other sort of work I'd rather be doing.