Whether you are truly interested in PM&R, or it's your backup as ihaveaquestion is implying, the question you asked is still a good one that may help others down the line. So here's my 2cents:
Not having a letter of recommendation from a physiatrist is sometimes okay. It does make us wonder why, but the right excuse can be acceptable. If, for instance, you were at a med school that didn't have a PM&R program or a PM&R department, and so your exposure to physiatry was from private practice docs in the community; nothing formal, but you enjoyed your time shadowing them and picking their brains (yada yada yada). In that case, not getting a PM&R letter of rec is understandable.
However, if you did a formal PM&R rotation for greater than 2 weeks with the same attending, and don't have a LOR from them, that will certainly raise the red flag to us.
OR, if your application with LORs was submitted to ERAS in Aug, but you just had a PM&R rotation in Oct, then of course that's understandable too. But in that case, I'd try to get that letter and bring it to any interviews you happen to get.
Now, just like any specialty, don't expect that kind of leeway from the top (more competitive) programs.