PM&R is slowly getting more recognition and popularity. I believe one day it will be very competitive field. I would break it down like this:
1) Lack of recognition, knowledge of what PM&R docs do. 99% of the time when I explain what I can do as a PM&R doc, fellow med students and some attendings are surprised. Some docs have the mistaken idea that PM&R docs are the equivalent of glorified personal trainers. They don't realize the amount of medicine PM&R knows, or how broad one's knowledge base must be if they wanted to practice say pedi PM&R and inpt + outpt adult.
With this lack of recognition, there may be a lack of prestige to the field like matching into a highly competitive residency. To some people, the prestige of a field is really important. To me, practicing good medicine and helping others is foremost, and I don't care very much what other specialties think of me insofar as it will affect referrals, etc. I think this recognition is changing as more ortho/NS practices have PM&R docs on board.
2) Average salary is not as high as other specialties. I think no matter what, the average salary of a PM&R doc nation wide is not as high as other specialties such as derm, radiology, radonc, gas, etc. Of course there are many PM&R docs doing extremely well, but I'm talking averages here. There also seems to be a Lack of knowledge of salary information. I've heard some say expect to make $170K starting in private, and others say $220K. Who knows? Salary was not a huge deal for me, but I know when many of my classmates asked me, they seemed disappointed and chose fields like radiology or neurosurg.
3) People don't realize the hours we work until they themselves are in residency. And then they switch. I have heard of almost every single possible specialty switching into PM&R. I have heard of residents in ortho, gen surg, anesth, radiology, IM, ob/gyn, etc switching into PM&R.
4) Many do not want close patient contact, an integral part of PM&R. They hate clinic work/inpatient wards, they don't want to think of social issues or have family meetings, etc. I think you have to have a strong interest in your patients to do PM&R and be patient. I have gotten the comment from friends who told me honestly, "I don't think I have the patience for a field like PM&R."
When all is said and done, the future of PM&R looks bright. Some residents or practicing MDs congratulate me and say "excellent choice" about my decision.
There has been a move towards more outpatient based work in musculoskeletal medicine, which generally has allowed PM&R docs to be at the forefront and the first MD a patient will see. I think more students are recognizing the advantages of this field. There are some who think the field is 'easy to do,' but I can assure you that aside from hours, there is a lot to know just like any other field.
[On a rambling side note: Ironically, the 'easy to do' deterrent doesn't hold up when it comes to many popular specialties such as derm (which I have friends doing and they readily admit it is not rocket science or as broad as say a field like FP or med-peds). In the end, I suspect salaries to be the most critical factor (sadly) as this would be an explanation why something like derm is so popular. However, you can choose to specialize in a subfield that pays well in PM&R also, like EMG.]