It's a balance. In my personal opinion, leisure time is just as important for getting into med school as studying. If I studied all the time, my barin would melt, and there would go all my hopes...I live by the philosophy, "Work hard, play hard." I do have leisure time, but I plan it into my schedule the same way I plan in my study time. As long as you don't consistently and haphazradly add hours of fun where hours of studying should be, you'll have room for plenty of leisure. There will be a week here and there when there is no time for leisure (I once had 5 finals in two days; one of them was orgo and another was cell...Sleep was my only playtime that week), but you'll suck it up and crash for a few days afterwards.
But that's just my experience. There are some people out there who really struggle with the material and can't pass science classes without studying all day every day. You'll know if you are that type of person after the first year, and can decide then if it is worth it to continue on as a pre-med or not.