Not that I disagree with the general consensus here, but just a personal anecdote:
I currently have a clinical mentor who is a surgeon. The first time I met with him we had a quick phone call and I ran across the street to meet with him. (I am doing the graduate portion of my dual degree--he knew I was coming from lab). I showed up in khaki pants, and a short sleeved polo shirt.
He was pretty quick to comment. He mentioned that when he did his MD/PhD that things were very casual with his graduate mentor and he could wear whatever in lab...but stated, "You have to be able to adapt to these two different worlds. If you are in the hospital, you should have on a tie and white coat. You have to look the part." So, whenever I go to the hospital, even if just to meet with his nurse for 2 minutes to pick up some papers, I look the part. Indeed, there are individuals who will frown upon lack of tie and white coat, even if the majority will not.