It's a personal choice. Personally, I graduated a year early with 2 majors, and I've found interviewers to like the fact that I graduated early (haha, they think it's a reflection of how smart I am, but in reality, I was just lucky and got some good advice on planning my class schedules) Although, so far I've only had one person ask about it.
I think you can back up graduating early. My primary reasons were: I don't need any more debt going into med school! And that I was offered a great job that I couldn't really turn down, which offered much more experience than another year of classes would. (And that I had a nice upward GPA trend that I didn't want to jinx, this obviously wasn't a reason I ever gave to an interviewer).
I think as long as you can convey your maturity at an interview, and don't graduate a year early just to party (since they will ask you what you're doing with that year), it shouldn't play a huge factor one way or another I don't think. Good luck either way though!
Just to clarify, you probably shouldn't try to matriculate straight after 3 years of college though. Since you'd have to apply summer of sophomore year, and you'd be competing with people who've taken harder classes, and had more EC's than you most likely.
Just my two cents.