Points system to rank applicants?

[MedicalMannic]

---

Members don't see this ad.

I've heard through the grapevine that some programs use point systems to rank candidates i.e. assigning quantitative values to steps scores, publications, AOA, etc.

Is this true? Do programs really check boxes in this way?

 

[Merovinge]

Some do yes, that practice is more common in more competitive specialties (when I was a med student I had heard the majority of ortho programs were ranked like this). Ultimately most program generate a list from top to bottom based on some kind of system and then move individuals either based on points or committee meetings or both following the interview day. Is there a different system you had in mind?

 

[FlowRate]

Think about it this way. You received 1000 applications for a residency class of 10. You can interview 100-200 of those applicants. You also have other administrative and clinical duties and only a month or two to screen applications. What do you do?

Whether you use an explicit ranking algorithm, cutoffs, or whatever, you're likely to start with something quantifiable (or binary--foreign? DO? failures?).

So now you've narrowed it down to 200-400 applications. Time to read personal statements, MSPE's, and LOR's. Maybe you'll have the people doing this step create snapshot summaries of each of those things. Maybe you'll find a way of rating those factors.

Some places will continue to tally up a numerical score based on the points reviewed so far.

Now you're interviewing people. What do you do with the interview? Probably rate people again. Another number added to the tally.

You have to list some number of residents on your rank list. How do you start? It's probably easiest if you put them in order. Maybe by that tally you've been keeping. Maybe just by their interview reviews. Maybe the PD has their own running rank list that they keep in their desk drawer made of magnets and ERAS pictures and hot glue.

Then the recruitment committee reviews the rank lists and makes adjustments. Maybe senior administrative staff are allowed to bump a few of their favorites up a bit. The PD probably further edits the list based on what the ouija board says.

The above is an amalgamation of my knowledge of processes are various places. I imagine it's generally representative of the idea of what most residency programs do. The permutations and specifics are different at each place but the basic idea is that you have to cut down your applicant list, generate an initial preference order (interview invites), and then update that preference order after you interview candidates (rank list). That's best accomplished by doing some amount of quantification.

 

[BigJaySon]

This process is so effing stressful 🙁

 

[DisorderedDoc417]

I've heard through the grapevine that some programs use point systems to rank candidates i.e. assigning quantitative values to steps scores, publications, AOA, etc.

Is this true? Do programs really check boxes in this way?

Yes. I wouldn’t really waste time trying to “figure it out”. Some parts of it are objective (scores, achievements, etc.) and some are subjective (personality, interview style, overall impression of the candidate on interview day). Good goals would be to do your very best on the steps, coursework, etc. AND be polite, genuine, and friendly with those you meet along the way. Subjective qualities still go a long ways in psychiatry, and there are plenty of people that have all the numbers but not enough sense to treat everyone with respect on interview day. Don’t be that person.

 

[TexasPhysician]

I've heard through the grapevine that some programs use point systems to rank candidates i.e. assigning quantitative values to steps scores, publications, AOA, etc.

Is this true? Do programs really check boxes in this way?

Look at it another way (simplified version):

You’ve got 6 job openings and 900 applicants. You are only given 15 interview days where staff are available to interview 5 applicants/day. You alone must choose 75 to initially invite of the 900. Each of the 900 comes with pages of data. I’ll give you 10 hours to choose your 75 from 900. No help is provided beyond a computer to narrow your decision.

Each applicants interview with 4 of 10 faculty. Many faculty don’t have access to subjective interview data as applicants only interview with some faculty. Each faculty is given 15 min to read and prep for each interview.

Interviews are spaced over 4 months. Another month later to recall data from all 75 applicants that you haven’t seen for up to 5 months.

How do you reasonably sit 10 faculty together to rank 75 applicants with each faculty having limited data? By the way, I’ll give these 10 people 3 hours to finalize their list.

The above is not exact, but it gives you an idea of the difficulties involved.

 

[EarlyCareerAcademicPsych]

At our program, where I have done a lot of the ERAS screening, we get over 1300 applications for 11 spots.

I try to look at all of the applicants because I do not believe that USMLE scores have helped us determine who will excel as a trainee in psychiatry (and - then - who will excel as a psychiatrist).

The most important factor is interest in psychiatry as demonstrated by SOMETHING. This can be as little as a sub-internship (which not all applicants do) to as much as one (or more) of the following: clinical research in psychiatry, leadership in the Psychiatry Interest Group, Mental Illness advocacy.

Unfortunately, there are a number of medical students who apply in psychiatry as a "back-up" or because of "lifestyle factors". I am not intersted in interviewing these candidates. The best way that I know to screen them out is to look for activities that demonstrate interest. If you've got one of those, then you are likely to get a chance to interview with us.

 

[NickNaylor]

At my institution, initial invitations to interview are provided largely by the PD after reviewing applications. I believe the APDs also review applications given that we receive a very large number of applications - something like 1000-1500 - in a fairly short amount of time. However, this initial review is strictly qualitative.

When it comes to reviewing applicants after they've interviewed and creating the final ROL, we use a rubric scoring system. We look at a variety of factors, each of which is associated with a certain number of points. Some of these things are relatively black and white - e.g., this amount of research productivity results in X amount of points for the "research" section - but some things are qualitative. In our system, the most important things by far are interview evaluations, clinical performance, and, to a lesser degree, pre-clinical performance. There are several other criteria that we use, but each individual criterion is weighted much less compared to these factors. At the end of the interview season, applicants are entered into the program's ROL by order of overall score. The PD will rarely make a small correction to the score if he feels that the rubric doesn't adequately capture his overall gestalt impression of the applicant, but he is quite open about this and the correction is comparatively small. Rarely does it result in a meaningful difference in the overall outcome.