It has been my observation while on the interview trail for surgical residencies that yes it does depend on the program, but it also depends on the type of residents they attract. One program, which I will not identify here, seemed to thrive on its reputation of being hard on hours and fairly malignant. The residents almost seemed to enjoy complaining - even when some of us were doing more hours than they! Seems that its "macho" to complain about how much you work, and how hard you have it - and frankly, some people like that.
The hours are long, but then they are for most residencies. The work load does vary by the program - I know that seems like a cop-out but frankly it makes a big difference if you're at a county hospital or VA that has no ancillary staff after hours or on weekends (so you have to do all the scut yourself), or at a community program which has call q4 and some which have it q2, but then there's no cross cover.
Most of the stories you'll hear are anecdotal and will have little relevance to your experience. I could tell you the story of a friend I have who dropped out of his surgical residency because he never saw his family (he's doing Peds now), but that doesn't mean that will happen to you. Being organized helps, and choosing the best program for you does as well. I recall seeing some books written about being a surgical intern, but they were always about people at places like BWH or some other hotbed. No one ever writes about being at a small town community program with little trauma experience - maybe I'll be the first!