Agree with the above.
Try to avoid going on interviews during a sub-I. On other rotations, you should be able to leave to do interviews if you need to. The attending and house staff and fellows should NOT need you to do work. You are a 4th year med student...you are a student...THEY are the doctors. You are not getting paid. You should be willing to work very hard when you are there, and should pay attention and be nice to your patients and do your work. However, IMHO it is very silly for someone to hold it against you when you have interviews. If you are on a rotation where you do actual work, such as a consult rotation or inpatient medicine or surgery, it would be best to give them advance notice when you are going to be gone (i.e. don't tell them the day before). Sometimes attendings forget stuff, or forget to tell the rest of your team, so it would also be good to tell the senior resident(s) or fellow(s) when you are going to be gone.
This issue may rear its ugly head in the future. For example, when I was doing IM and had to go on a bunch of interviews for cardiology fellowship, I got some undeserved crap from fellows on the consult services I was rotating through, who couldn't understand why I'd be missing multiple days on the same rotation ("So, I guess you scheduled all your interviews during OUR rotation?!"). Depending on the specialty or subspecialty you chose, large numbers of interviews may be necessary...that's just part of the game. Don't let people intimidate you into not doing what you need to do. Try to stay in their good graces, too...I like the idea of offering to do makeup days. For future 4th year students, it's a good tip to schedule easy rotations in November and December, maybe October or January too, depending on when interviews are held for your chosen specialty.