but Penn pretty much has a round table right after each interview
I do know how UMN does their interview piece, but I would imagine each school does things differently.
Interviews at UMN are done by two interviewers. They lean toward closed file interviews, meaning the interviewers have not reviewed your application; they don't know anything about you prior to the interview. Because of the way it was explained to me I suspect that's not a hard and fast rule. (I asked at my interview, and they said "We try not to review your file beforehand, so don't assume we know anything about your background.")
They have a packet of info with them; each page has a quality/trait that they are evaluating along with applicable behavioral questions. Not everyone necessarily gets exactly the same questions. How the interview is conducted - who asks questions, whether they rotate, etc. - is up to the interviewing team. Some teams will interview a few candidates, some will interview a few dozen.
When the interview is complete, the interviewers talk through it to arrive at a consensus score on a scale of 1-5 for each 'trait'. It is not an average - it's a consensus. If they can't arrive at a mutually agreeable score, there is a multiple-step arbitration process that can, if necessary, actually lead all the way to re-interviewing. IIRC, the person who explained the process to me also said she's never seen that happen, though.
Once they've arrived at a consensus for each trait, it's a simple matter to add up the numbers to get your final numerical score.
I do not know if the interviewing team submits more than just a score; for instance, whether they submit their notes and observations as well. I don't know if they weigh certain questions in the interview more highly than others.
Either way, those numbers go back to the admissions director/staff, who tally up your academic score, 'subjective' score, and interview score. It must be a very quick process at that point, because they let us know, via email, within a week after the last interviews.