Nobody here is singling out any specific program, but residency is an individualized fit with a lot of egos involved. Depending on where you do your clerkships, you will probably come to realize that there are some programs that are hard because there's a ton of patient-related work to be done, but sometimes programs are just hard for the sake of being hard. It can be a trickle down effect from the director/attendings/chiefs ("my first year was hell and my attendings were strict, so now it's your turn").
At most historic "top programs" you will find a rigid structure (academics, rounding, cadavers, research, etc) and residents who are run fairly ragged. Residents then have almost no choice but to work a ton and graduate the program as an outstanding surgeon (or get fired along the way). It really is a military-type of rank system at some programs, and that means some of the insults, orders, duties, etc can be intimidating and tempers run short (it doesn't really make you Mr. Friendly when you sleep 3-5hrs per night with barely any days off). The downside of intense training is that you could become pretty miserable and have a tough time taking care of your family/health/sanity/etc if you don't realize what you're getting into.
It's like that old saying "no matter how hot the girl, there's a guy somewhere who is sick of her BS." Well, same goes for residencies...
Naturally, some good students now try to find a program with similar numbers and opportunity as the elite programs... but with more flexibility. However, that flexibility means it's up to each resident whether they want read, research, etc to graduate the program as a great surgeon or whether they want to do the minimum and end up just average. At most residencies, just like most classes, the efforts and interest level of the person learning matters as much or more than the efforts of the person teaching.
Tough programs with famous attendings and successful almuni will always get a lot of interest, though. They are famous because they've gotten results and obviously give you what you need. I'd say the majority of people who are in professional school are at least somewhat type A personalities who like structure. With the podiatry graduate:residency ratio swinging back in favor of the programs pretty soon, I really doubt you will see any dropoff in interest for any program that has good numbers and/or history of successful grads. The key is just to find the residency that is the right fit for you personally (structure, faculty, types of cases, co-residents, location, etc).