Your right to a certain extent. There are many practices where they refuse to issue narcotics. The purely do interventions. You also have the right to refuse someone who is being abusive in your waiting room. However, it is not always as easy as just 'cherry picking' your patients. There is a certain obligation as a physician to stand by your patients. The nature of pain, as has been illucidated here in previous posts, has a definite psych component even in baseline normal individuals. By definition, the practice of pain medicine has a different feel than anesthesia. It is a clinic based, continuity of care field. Long term management of patients will inevitably lead to issues that are unique to this relationship. Even though you can taylor your practice to minimize this exposure, it cannot be avoided.
This is a reality, but not necessarily a negative. We are all looking for different things in medicine. Some have come on here and asked if IM paid the same with similar hours would we do that. A legitimate question, but insane to me personally. I love anesthesia, and even though the hours and salary are sweet, I like the nature of what we do. Others miss the traditional role of 'physician' that is demonstrated by primary care docs.
Pain gives a certain sense of autonomy that is missing in anesthesia. As anesthesiologists we are essentially consultants. We offer a service, and consult on others patients. While this is definitely a plus in many people's book, some miss having their own patients. Pain can fill this void. You will get varying degrees of opinions regarding this topic based on personal preferences. Currently I enjoy the consultant life, but many of my colleagues are already looking at pain.