"You chose this job" is the last bastion of the desperate. It is the poor man's rationalization for pathetic performance. Non-ED physicians are not angry about seeing patients. They're angry about seeing patients that have been evaluated in a half-assed manner (if at all, on occasion), with an incomplete workup, and a shot-in-the-dark diagnosis. Is this every patient that the ED gives out? No. But it's a substantial minority, and at every institution. And the typical excuse is "you chose this job ...so suck it up and take the admission." Hey, just because YOU think the person should be admitted, doesn't mean I can't evaluate the patient and decide otherwise.
I guess your "suck it up" mentality is a one-way deal (unsuprisingly). When the ED is doling out patients, everyone has to "suck it up", but when the other services decide to refuse the patients or want a better initial evaluation it's "hey, hey, hey, that's not my job!!" This is why people hate the ED - because of putzes like you. When people come and refuse patients, why do ED physicians fight so hard? Shouldn't you "suck it up" and deal with the patient because "you chose this job"?
And don't even try to compare the ED to clinic patients. Patients in specialty clinics (Surg, IM, etc) may have uncomplicated problems - as do many ED patients. But the ED just deals with one thing: the CC. In clinics, you deal with the CC, then work your way down the entire PMH (in terms of preventative care, follow-up care, etc.). Do you guys see a lot of patients? YES. The reason? Because you're not by any means treating the patient or dealing with chronic problems (which are far more of a pain in the butt that dealing with an acute issue).