Honestly I know attendings in pretty much every specialty with both good and terrible lifestyles. I'm not sure if there's a hard stop avoid list. Certainly things like Neurosurgery and the other surgical fields tend to skew more toward longer hours, but plenty of people are able to build practices that aren't quite so bad. The key differentiator early on is location. It's much easier to negotiate better conditions when you're a hot commodity rather than trying to be the umpteenth -ologist in the Bay Area.
Another key factor is practice environment - academic, private, employed, etc. Each with plusses and minuses, and each having a major impact on lifestyle regardless of specialty. There are plenty of partners in big private groups making serious bank working very reasonable hours with no call, but this is often more about their ancillary revenue streams rather than whatever field they trained in. Conversely, there are academic and employed docs who are still mainly paid for their productivity and so they may have to work harder for longer than their peers who are developing other revenue streams in their private groups.
Another key issue is just how hard someone wants to work. EM is traditionally thought of as a good work-life balance, but I know plenty of EM attendings working extra shifts to pay off loans or retire early, especially those without families yet. I know a couple of neurosurgeons who both busted arse, lived on a shoestring, and retired at 40, so while their lifestyle sure sucked in residency and early attendinghood, now they're not working at all and just travel. Nobody would ever argue that neurosurgery is a cushy lifestyle field, but I know of at least 2 mid-40s neurosurgeons who work zero hours a week.
So yeah, very complex question! All of these questions ultimately boil down to looking primarily at what makes you happy, what you can tolerate and what you can't. I would hesitate to write off any fields until you have your short list of options and then see what's possible from those.