If you have the money, go back for a day outside of the "interview" stuff. That way, you won't have the pressure of the interview and you might get a better impression of things that are most important to you. Relying on what you "hear" from others may or may not be accurate. I can also tell you that I didn't make my rank on the basis of less than 24 hours in any one place. I did have the advantage that I didn't apply to places that were far distant from my location so that I could go back outside of the interview day. Every place that I interviewed offered this opportunity and I took full advantage of it.
I can tell you that "reputations" of various programs change from year to year with each incoming residency group and the hiring of new attendings and other administrative changes. You have no control over any of these things.
I can also tell you that unless you are in a program, you can only speculate as to the day-to-day events, learning esperiences and other things that will affect you if you match there. Again, unless you are a resident in a program, you are hearing about a program through the eyes and ears of someone else.
Go back, if you can, and use your own senses, impressions and information gathing. Try to spend some time with the interns (they will be the closest to you if you match there) and get a better sense of how these programs work as they have different environments, locations and other things. In the end, you want to make the best decision that you can.
