Hey, I had to respond to this since back as an MS3/4, I struggled between choosing anesthesiology and pm&r briefly, but chose pm&r because i really saw it as something I could be passionate about vs anesthesiology was more just interesting and very pathophysiology oriented, but i didn't like the OR setting. I too want to pursue pain.
You definitely have to do what you see yourself liking the most. Keep in mind that PM&R residency is a diverse training period where you will do work with SCI, TBI, CVA, burns, amputee, sports, in addition to pain. It will definitely feel a lot like inpatient medicine on some wards where the patients aren't even good rehab candidates and are basically like a step down ICU. So you have to work in these diverse settings, and can't streamline it to do only pain related work. But if you like it, go with it. 😀