Amyl,
It really comes down to which base specialty you enjoy the most. There are three roads to pain medicine: Anesthesiology, physiatry, neurology/psychiatry. For all intents and purposes, anesthesiology and physiatry are the two MOST recognized in part because the base specialties incorporate the most pain management education in their training.
Don't choose a specialty based upon what you MIGHT want to do for a sub-specialty---your opinions and desires will likely change as you progress through your training. Do a rotation in each and see which one you think "fits" better. Anesthesiology is more acute care oriented; physiatry more chronic care oriented. Anesthesiology tends to be more technical and procedural; physiatry a little more cognitive and intellectual. These are just broad generalizations.
Also, consider other sub-specialty opportunities that will available through each route. Anesthesiology offers critical care opportunities, cardiac, regional; physiatry offers sports medicine, neuromuscular, and musculoskeletal medicine.
You'll encounter many STRONG opinions on this topic. Take it all in stride. Best of luck.