Grades are funny. I graduated with about a 3.7 (honestly, I don't remember exactly what it was anymore) and only worked part-time (~15-20 hours as an RA) as a senior. However, my senior year was easily my best year GPA-wise (over 3.9) even though I had more to do that year than any other (disclosure: I worked for a year after undergrad and applied to medical school at that time, so I wasn't traveling).
I think most people can probably handle a part-time job fine in undergraduate, but full-time may be a different thing altogether.