I've actually done this after my last two interviews. I've been a total mess at them in most regards. I stayed with med students the night before. They said that they couldn't be back at their room before I would have to leave, so I couldn't leave my bags there. Thus, I would walk into the admissions office looking like a pack mule in a suit (I was carrying a suitcase, briefcase, backpack, garment bag, and a sleeping bag). The secretaries in the offices were very nice about it as they had seen the same thing many times before and readily showed me where I could place my bags. After my interviews and the tours and stuff, I didn't see any problem changing back in to street clothes. The interviewers were away in their own departments and offices, so I didn't have to worry about bumping in to them that much. And the admissions staff were usually pretty cool, they knew the drill pretty well and didn't seem to mind. So unless you interviewed with the dean of admissions and you were going to run into him again before you left, I don't think there'd be a problem with changing.
