Literally 6 years ago the answer to this same question when I was entering the match was undoubtedly "EM". Today the answer is "Anesthesia". Tomorrow it will be something else. All of that combined with the fact that the entire US healthcare system feels like its on the verge of collapse. There are no right answers. Tomorrow any given specialty may change, the job market will dissolve, midlevels will takeover, etc. Nobody predicted what would happen to EM, and nobody can predict when or whom the next victim will be.
Both rads and anesthesia have had "doomsday events". EM is going through ours. Just like nobody could predict how poorly EM would be doing right now, we are equally bad at predicting how well we may do in the future.
I would caution against picking a specialty, like EM in particular, only because you are "afraid to lose your medical knowledge". The truth is, when you graduate medical school, you have absolutely no medical knowledge. There is nothing to lose. You can only gain the skills and experiences of doing something that you enjoy doing. If that's replacing 10 hips in a row in the OR on any given day, that's great. If that's seeing the hodge lodge of complaints that comes through the ED, great. If that's doing heart transplants at 2AM, more power to you.
Everyone in medicine is a specialist, including EM. We specialize in undifferentiated patients. We specialize in initial resuscitation. We specialize in disposition. It doesn't sound as tangible as specializing in "joints" or "corneas" but it's still a specialty, even if others in the house of medicine don't look at us that way.
EM is a great field that right now has been completely exploited and gutted by CMGs, midlevels, and C-suite dirtbags. There are some folks, namely McNamara at Temple and others who are leading the fight to take us back to the glory days, but it'll take awhile. I hope we get there eventually.
My decision to go into EM was very much shaped by the fact that I looked up to the EM faculty at my home program in medical school. These people were like Gods to me, and I thought if I went into EM I would be more like them. That was short sighted, and I wish someone told me to focus more on the bread and butter, the patient population, the day to day work as opposed to the heroic saves by these mythical figures. It really demonstrated my lack of maturity at the time. This goes for any specialty, not just EM, it's very easy to meet a "cool attending" and want to emulate them and as a result go into their specialty, which may end up being a grave mistake.
I did briefly consider trauma surgery, so I guess in a way, I dodged a bullet.