In academia, pretty much everyone is salaried. That is, if I see 10 patients in clinic I get paid X... and if I see 20 patients I still get paid X. Some attendings might have a productivity bonus, but certainly none of the residents do. This, plus the fatigue from your job juggling a million things, leads to you having an incentive to do as little work as possible. You also have incentives to do more work of course, but those are typically more nebulous (as in, you have an incentive to get the best education you can, but that is a lot harder to quantify today relative to an extra hour of sleep).
This leads to a dislike of anyone that gives you "extra" work. And the ED seems to do nothing EXCEPT give work to all of the other services. If a patient comes with chest pain, the ED takes a careful history, does a physical, EKG, and a set or two of enzymes and sends them home? Well, you never hear about that patient. So the only interaction you get with the ED is when they want to give you more work. This leads to some resentment.
You can complex that with the fact you know your field better than they know your field. That is, Emergency Medicine physicians are the experts in their own domain: The evaluation and stabilization of the undifferentiated patient. But they aren't the experts in the management of just about any condition once it's stabilized. Their job is to get the patient to the actual experts... and often while accomplishing their task, they don't do things in the same order or in the same exhaustive detail that the new primary service would do them in. Most of the time, it doesn't make a huge difference, but you can certainly look down on them for not knowing the nuances of your specific field if you aren't careful. Of course, you can only do that after they figure out the problem falls in your wheelhouse, in between seeing the other half dozen fields worth of patients in that few hour period they were seeing your patient.
The "ED just gives me work" problem tends to get better if you ever end up in a non-salaried (private) environment, but persists if you're in a salaried environment (whether in academia or directly hospital employed without a production component). The "ED doesn't know all 40+ specialties worth of medicine" problem never goes away.