I remember a medstudent asked an ER doctor about his job. I'm paraphrasing because I don't remember the exact words but it was something to the effect of....
-I like my job, but I usually get patients are not thankful, they see me as a servant, anytime I call a unit to call a patient they are rude to me because they don't want a new patient, and whenever I deal with other departments it's usually due to a turf-war.-
Every field has pros and cons. My dad is surgeon. He's pretty much at work all the time, even when he's home. He gets beeped all the time, and when he's home, he's still in work mode, more concerned about his patients than other matters.
IMHO the benefits of psychiatry, if you like the field, far outweigh the cons. You can build your hours to pretty much whatever you want (a mere handful a week to whatever upper limit you want), high flexibility in practice types (emergency, outpatient only, inpatient, consults, private practice) and if you do a good job people will come to respect you over time.