I wasn't "knocked off" by college, but it definitely was a new experience for me.
I breezed through HS, and didn't do a damn thing. I had a 3.8 GPA, regular classes, no struggles. The only C I ever earned was in biology. Go figure.
I wasn't even concerned with my grades. I rarely studied except for the 10 minutes before the test and didn't put in considerable effort at all.
I graduated early at 16. I didn't even take all the classes. The dean of my school said i'd probably end up doing fine for the remainder of my HS years, so she gave me the credits and I went straight to college at 16.
At college, for the first time in my life I studied for a test, had to review daily, etc. I was worried and scared because I had no idea how I would perform. This turned into an extreme form of paranoia and guilt. I'd study hours and hours every day and obviously did very well my freshman year. I earned a 4.0 both semesters. After my first year, I got the hang of things. Now I know what I need to do, how hard I need to study, etc.
After the initial transition period, you should be okay provided you have your priorities set. There's no need to worry about the difficulty of medical school at this point. You can't compare something 4-5 years down the road with any initial troubles you are having now, just out of high school.