For most students the most significant academic impact you will see between class sizes will probably be wrt clerkships and AOA.
You should ask clerkship students how many med students / team or clerkship site on a given rotation. During normal times at our smaller school with a fairly large number of clerkship sites this number is typically ~1 MS3 and if there was another student it was usually an MS3 + MS4 on Sub-I. At least here many perceived that to be advantageous to them because they rarely felt like they were "competing" with another student for attention, cases, patients, or on evaluation. That said, it's perfectly possible to have a similar experience at a school with a much larger class size if it also has many clerkship sites / a very large medical center.
Depending on how the school does AOA, class size will likely have an impact with it potentially being harder to land at the "top" of the class by whatever metrics they are looking at in a larger class.
Socially, it will depend on what kind of environment you prefer and the school culture.