I was planning on going into psych, most likely addiction psych, for most of medical school until and end of my psych rotation. The rotation is what guided me down that road, and I rotated through almost every aspect of psych (inpt, outpt, consults, geriatrics, inpt rehab, outpt rehab).
To me, it psychiatry was a lot less subjective than I originally imagined and had more algorithms/criteria for diagnosis than appealed to me. I thought it was more "easy going," but the fact is psychiatry is a medical specialty, and the way it approaches diagnosis reflects that. But I just didn't enjoy the art of diagnosis in psych. I did really enjoy talking to my patients and getting to know their stories, but in general I had to work faster than I wanted and I never got to really focus on the patient as much as I imagined I would.
It also seems very strongly the future of psychiatry is more biological--more medication, more medication, less talk. I didn't like that at all, but I think it's a reality considering the shortage of psychiatrists. Psychiatrists are best at the medical aspects of mental illness. There are others who can handle the counseling/talking part much better and more cheaply, so the days of psychotherapy, in my attenedings' opinions, as well as my own, are coming to a close. With the exception for cash-based private practice, but that goes against everything I went into medicine for.
I've definitely met psychiatrists who say that psychotherapy is still alive, and I'm sure pockets of it will remain, but it's going to be much more difficult to keep your practice open and make enough to pay down your loans if you keep doing talk therapy. Now, if you've got no loans and don't care about your income, I think it'd be really rewarding...
Lastly, while I still liked psych, I just didn't love general psych. I did love working with some patients, but the field didn't energize me as I hoped. I found addiction psych the most fascinating, but I could tell I was going to get jaded if I went into the field. I think you really need to focus on what your personality can handle and thrive in (ie., can you really tolerate your young children dying in peds heme-onc?)
Plus, I found out about PM&R at the end of the rotation and that connected all my loves together--interest in the psychosocial needs of the patient, neurology, non-operative orthopedics, biomechanics, long-term care, focus on improving quality of life. It just seemed like a better fit. When I rotated through it the patients energized me, I loved the science behind the specialty, and the people. (Psychiatrists are pretty cool too. Some are a little cooky. But I thought that made it a bit more fun as well...)
Still, I think psych is a great field, and if I couldn't go into PM&R, the specialties I'd consider first would be FM and then psych. Despite everything I said, I think it's still a great field where you can make a huge difference in people's lives, helping others that are often shunned (even by the medical field). What it came down to for me is psych moved down my list because it wasn't everything I was thinking/hoping it would be, and PM&R was more. Well, that and I didn't even know about PM&R until the end of my psych rotation.
Anyway, you just gotta go with where you feel you'll be happy and thrive.