I work at a very busy psych residency. My worse months hours wise were still Medicine and Neuro, and I feel sorry for those residents. But as an intern the "worst" inpatient rotation was 7a-5p, with one or two 4hr calls 5p-9p during the week, and weekend call three weekends consisting of either a 14hr single day, two 5hr days, or a 9p-7a Saturday night.
Overall I got to work out 3-4 days per week, see my kids most days (even on call days I'd come home for dinner if I could), have at least a day off on the weekend. The actual work days inpatient were busy until 2-3pm but then usually died down. Some residents would alternate being "on" that day and the other would leave early and be available in emergencies. I usually hung around and got all my reading and stuff done so home was home.
I'm sure there's some gem residencies out there, and EM seems like the hours are similarly low, but psych really does have the best hours by far.