In terms of overall public health, I would exchange needing a referral to go to the ED for everyone in the country actually having a PCP in a heartbeat.
The ED saves lives, but they suffer from not knowing anything about the patient. I would venture that the vast majority of work they do could be handled just as easily, more cheaply, and more effectively, by a PCP that knows the patient.
Most of the "emergencies" in the ER are better described as urgent care (if that). They would be a ghost town if they only served heart attacks, strokes and the like.