I understand your anxiety on this. I asked the same questions 8 years ago when I was applying, because being from an EM background and having been out of residency several years already, I had no letters of recommendations from any anesthesiologists, physiatrists or IPM physicians when I applied to fellowship. I ultimately bit the bullet and decided to get letters from people who knew me and my skills best: My Emergency Department Director, EM residency program director and a fellow EM doc (also Chief of Medicine at the time), all of whom I worked side by side with for nearly a decade. Ultimately I got in, but the fact that I had no letters from anyone in the Pain or Anesthesia world did leave me with a sense I was at a disadvantage.
My advice would be to get a couple of letters from people in the PM&R world who know you really well. Those letters sound best and are usually more persuasive anyways. Then, if you can, get one token Pain or Anesthesia letter from someone you may not know as well, but only if they can write you a decent one. If it's going to say, "I don't know this PMR dude/dudette who shadowed me for 5 seconds. -Sincerely, Famous Anesthesia Pain Guy" then you're better off sticking with sincere, quality letters from people that may not have the pedigree you wish they had.
Also, since there clearly is a bias against all non-anesthesia applicants, my advice is to try and overwhelm those odds with volume, by applying to every program in the country. As a non-anesthesia applicant, you cannot assume you're on a level playing field, until and unless you've been accepted to an ACGME fellowship. All 80-90 programs. You can always turn a program down or refuse to go to a place you matched at, but you can't create a fellowship offer out of thin air, that you never applied to.