I shadow at the James A Haley VA, and all the doctors I've ever shadowed have been sooooo nice. I also coordinate getting shadowing opportunities for students via help from the volunteer coordinator there.
Anyhow, I'll tell you something. None of the people that I have set up at the VA have ever had problems. In fact, the doctors always treat them very well, the patients never seem to mind that someone is in the room observing, and actually on a couple of occasions, when I was shadowing the patient offered me a chair.
The patients also have talked to me and what not. Many VA patients are older patients and don't really care if we observe. But the doctor has never done anything illegal and told the patients that we are medical students.
Also, we get Hippa Certified through the hospitals volunteer services.
Other people that I set up in shadowing radiologists, cardiothoracic surgeons, radiation oncology, ophtamology, and cadiology, have told me that they have gotten to see a great array of things and none of the patients care that they are observing. The doctors also have said you know email me if you have questions or things like that.
The radiology and ophthamology people even got to see surgeries and what not.
But overall, I've never had any problems with patients minding or with doctors getting annoyed.