I found second year to be easier than first year for several reasons. 1) First year got WAY too bogged down in the details, and unimportant details at that. 2) First year was BORING and taught by people who had no idea what was important to future doctors (for shelf exams or boards, even). 3) Second year was when we finally started to break the surface on the kind of stuff I applied to med school to learn. 4) Most (or at least more than first year) of the second year stuff was clinically directed, which was motivating to me, no matter how tough it was.
Like phoenix said, third year is a culture shock. It's really difficult to directly compare or contrast the first half of med school with the second half, because they're just so different-- it's almost like earning two different degrees in medicine. The hardest thing about third year (to me) is having to work all day long 5-6 days a week. I eventually got used to it, but the first month was absolute hell for me, mainly since I was the kind of student who, during first and second year, skipped lecture if it ever started before 10am. So regardless of how tired you are, or how little you enjoy the rotation you're on, you still have to show up bright and early and act like you want to be there. And man, it can get really tough sometimes. And I think most everyone else would agree to an extent.
Of course, beyond the requirement to work so much, how "hard" or how "easy" third year is will totally depend on the individual student. If you love taking care of patients and learning about medicine as a whole, you're going to really enjoy third year, even during the hard times. If you don't care much for primary care, or patient contact scares you, you're probably not going to be all that thrilled, and at times might find some rotations excruciatingly painful to trudge though. Again, it all depends on the student, and once you're an M3, you'll find out soon enough where you stand.