I do think it depends what you define "sports-only" as. I found multiple non-operative ortho clinic only positions when looking for my job out of fellowship. If you want the team coverage/collegiate+ only position, then yes, those are rare.
As an aside, I wish there was a rebranding away from "sports." For every athlete I see, I see dozens of overuse/aging orthopedic issues. Yet the top FM/sports programs will tell you how much better they are than PM&R/EM/peds because they're better equipped to care for the whole athlete, leading to ongoing bias in the field. I'd argue those other fields are actually more prepared in many ways for the current non-operative orthopedics role.