Optometry school is similar to other programs (e.g., med, dental, pharm, etc.) in that the difficulty lies initially not in understanding the material per se, but having the skills in place and motivation to process a great volume of material. Some other people in this thread have hit on this topic, but I'd like to expand on it a little. Take the study of pharmacology for example. I'm in my 2nd year at OSU, and while I don't think learning the mechanisms for the drugs we studied was hard, it did take a lot of study skills to organize all 300 of the drugs that we covered last quarter. I guess it depends on how you got your grades in undergrad. If you were able to get good grades without studying really hard, you'll find that that simply is not possible in optometry school. You need to have your "learning style" or whatever works for you figured out so that you can keep up.
I was one of those people with stellar OAT scores and not so stelar GPA, so if you're coming into optometry school like me, I'd recommend seriously researching how to learn before you start school. The last time I studied learning skills was in 8th grade, so taking a refresher course in that wasn't boring! If on the other hand, I actually had been someone accustomed to working around the clock to get good grades in undergrad, optometry school probably wouldn't have seemed significantly harder than undergrad...that's probably a better approach!