I suggest that you find and read "What exactly can I do as hospital volunteer?" posted by NLS earlier this month. Many great responses were posted which may shed light on your question.
In my opinion, you have heard biased responses from disatisfied premeds who are not savvy enough to gain placement in positions other than clerical duties or are not motivated to seek them (you can decide which is more accurate for yourself). I tell folks to skip private hospitals and go directly to public/county hospitals where the need for help is greatest. From my own experiences, emergency rooms are a great place to volunteer. Working at a doctor's office is fine too. Volunteering at the right place where you can talk to patients and hear their experiences will start you on your journey towards becoming a truly compassionate and altruistic physician.