I've seen both. It's hard to generalize. I've seen really tough unopposed with heavy in patient schedules (great training though) and I've seen unopposed programs with really chill schedules, no running codes, barely do anything in the ICU, etc. It really depends on the program.
That all said, I would argue that the "difficulty" of a program should not weigh that heavily in your consideration of programs. First off, from what I gathered all residencirs are hard. Second, you don't really want it to be that easy, because this is your only chance to be trained and is the foundation of your practice throughout your career. Even if a program is hard, in FM, who cares? Its only 3 years.
Now none of this has to do with work-life balance, etc. Some programs might work you hard, but be very good about it to make sure you have a good balance with time/rotations set up in such a way for you to recuperate.