I can only speak from the perspective of a relatively small program (7 residents/year). The positives: everyone from all the years know one another well and therefore support and program comraderie is high. All the attendings and ancillary staff know all the residents, so first name basis (and feeling of a more casual environment) is the norm throughout the hospital. More flexibility in scheduling (we pick when we do our required specialty rotations, with which attending we work, and how we want our elective to be designed). I've noticed at the local university program that the "required" specialty rotations are scheduled for them because they have so many residents that there's no room for flexibility. Also more flexibility regarding when to have vacation (at least at my program). Possibly more flexibility in making changes in the program (this is an assumption I can't verify through) as there is less bureaucracy.
The potential downfalls: less people with whom to have an instantaneous social life. News about personal life can spread quickly.
Hope this helps.