Statistics of Rank Order List

[DrNick2006]

---

Members don't see this ad.

How many programs should be ranked? I think the dependent factor is what is the chance of making a rank list after you have interviewed at a place. I will call this the rank ratio, or RR. I will comment on how to estimate this RR in a moment. For now, assuming you know the average RR of all the programs you interviewed at, you can calculate your rank list length required to give you a certain confidence interval of matching. We'll asume 95% is acceptable, then the chance that you will be ranked at at least one program on your rank list at a 95% CI is described by the equation below:

(1) (1-RR)^n > .95 where n is the length of your rank order list, you can solve for n to figure out how many programs to rank. (obviously this is very dependent on how well you estimate RR)

A recent thread I read stated that the PD at BIDMC said during an interview that an applicant who ranks BIDMC first on thier list has a 50% chance of matching there. This means that BIDMC ranks half the people they interview, so the RR at BIDMC is 0.5.

Now, how should one estimate RR? Well, there are two considerations. 1) the competiveness of the program you are ranking, and 2) your competiveness within the interview pool at that institution.

as for #1, BIDMC interviews approx. 515 people for 48 spots, and rank approx 250, so they rank 5 times as many people as they have spots. BIDMC is a desireable place to do residency, so it is fair to assume that less competitive places have a longer rank list to spot ratio then 5:1. - meaning they rank a higher percentage of individuals than 50%, some programs rank everyone they interview.

as for #2, that should be based on your idea of where you fit into the pool, and is something we are all doing all the time when we visit this web site. Sooo...

If you are applying to competitive places that do not need to rank everyone to fill and are an average applicant at those places compared to others, than your RR should be about 50% given the example of BIDMC. Therefore, solving for n in equation 1, you should rank 5 places to give yourself a 97% chance of matching. If your a sub-par applicant compared to the other interviewees and you somehow managed interviews at the most prestigous institutions, your RR at those places may be much lower, something like 10%, BUT if you were good enough to get those interviews your RR at a less competitive program would probably be close to 1.

Of course all this is silly cause everyone matches in IM, and you can rank as many programs as you want as long as you like them, but I think the comforting thing is that realistically you only need to interview at 5 or so places to match and save time and money. This is probably a more applicable problem for comptetive resdiencies.

Sorry for this but I am bored on a very slow service --

Good day.

 

[Tar Heel Born]

Sorry for this but I am bored on a very slow service

Yeah, apparently so! That made my head hurt.

To answer your question, I'm anticipating ranking maybe 4 or 5 programs, depending on how things go on the interview trail (I'm doing either 8 or 9 interviews total). I'm not applying at any of the very most competitive programs (i.e. MGH, B&W, JHU, Mayo-Roch, etc.), but mostly top-tier others in the Southeast and West Coast.

 

[InternalWhizzzz]

that makes sense....

 

[aredoubleyou]

OMG. 😱

This is exactly why I am not going into IM.

Could you rephrase that into a way someone a bit slower (like myself) could understand, preferably without ratios and abbreviatrions. Or even better, how many interviews would an average candidate that is likely to be ranked, say at the 50 percentile at each program, have to interview at to have a 97% chance of matching? 5 programs.

Ouch, my head hurts....mommy!

 

[DrNick2006]

I was thinking.. you could always ask the program what percentage of interviewee's they rank, this is not an illegal question as far as I know, and you could get a very acurate estimate of your RR for each program, assuming random chance at your placement onto the institution's rank order list, the equation would then be:

1-[(1-RR1)*(1-RR2)....*(RRn)]

where RR👎 is the n'th program you interview ats individual RR

 

[DrNick2006]

An example:

lets say I interview at MGH and they say they rank 25% of the people they interview, and I feel I am an average candidate there, and I interview at BWH and they rank 30% , and then I go to JHU and they rank 35%, and then I go to duke and they rank 40% and the UCSF and they rank 45% of there interviewers, than assuming randomness after the interview process, my chances of matching would be:

1-[(1-.25)(1-.3)(1-.35)(1-.4)(1-.45)] = 1-0.112 = 0.89

so you have an 89% chance of matching. Notice this is not the same as taking the original equation and just averaging the RR's and plugging them in:

(.25+.3+.35+.4+.45)/5 = 0.35

(1-.35)^5 = 1-.116 = .88

but its close at low RR's and RR's close to each other. The closer you get to 1 or the more variation in your RR the less the original equation approximates.

 

[BigBadBix]

DrNick, this is a really interesting idea so I hate to say this, but it seems to me that there is an error in the logic from the original post. When BIDMC says you have a 50% chance of matching after interview, I don't think that can be directly interpreted to mean they rank 50% of their applicants. Your likelihood of matching shouldn't depend on how many they list but on how other applicants rank them and therefore how far down their rank list they go. Whether they rank 250 or 300 or 500 applicants, the only determinants of your likelihood to match will be how far down they go and how high they place you. So the info you would need to really know your chances would be the # of applicants ranked per available position at each program, right? Or am I missing something?

 

[Zolpidem25]

OMG. 😱

This is exactly why I am not going into IM.

Could you rephrase that into a way someone a bit slower (like myself) could understand, preferably without ratios and abbreviatrions. Or even better, how many interviews would an average candidate that is likely to be ranked, say at the 50 percentile at each program, have to interview at to have a 97% chance of matching? 5 programs.

Ouch, my head hurts....mommy!

just out of sheer curiousity, what are you going into?**

**actually, this was just to get another post on this forum, as I am working hard for 100+.

 

[Zolpidem25]

DrNick, this is a really interesting idea so I hate to say this, but it seems to me that there is an error in the logic from the original post. When BIDMC says you have a 50% chance of matching after interview, I don't think that can be directly interpreted to mean they rank 50% of their applicants. Your likelihood of matching shouldn't depend on how many they list but on how other applicants rank them and therefore how far down their rank list they go. Whether they rank 250 or 300 or 500 applicants, the only determinants of your likelihood to match will be how far down they go and how high they place you. So the info you would need to really know your chances would be the # of applicants ranked per available position at each program, right? Or am I missing something?

damn it, i was just starting to get it.... now i am thoroughly confused.

basically, what i gather from all of this is that if i interview at say 15 places. and i like say 12 out of those 15. and then i rank all of those 12 that i liked. then i'm basically virtually essentially pretty much almost without a doubt guaranteed to match, assuming that i didn't f**k up the interviews, and that i'm a decent candidate on paper. that's basically what i gather. for the most part.

 

[BigBadBix]

just out of sheer curiousity, what are you going into?**

**actually, this was just to get another post on this forum, as I am working hard for 100+.

You can do it, Zolpidem! If I can do it anyone can. I spent four years on here in the 10+ category. 🙂

Don't you love how we as fourth years have so much time on our hands that we end up setting SDN posting goals and deducing equations by which to estimate our chances of matching and such? Just goes to show what happens when you take a bunch of overachievers and give them too much free time. 😀

 

[aredoubleyou]

You can do it, Zolpidem! If I can do it anyone can. I spent four years on here in the 10+ category. 🙂

Don't you love how we as fourth years have so much time on our hands that we end up setting SDN posting goals and deducing equations by which to estimate our chances of matching and such? Just goes to show what happens when you take a bunch of overachievers and give them too much free time. 😀

Very true, I recently passed the 50 mark - it was a secret and sick little pleasure for me, but now that I know there are others like me I can come out of the closet.

I am going into anesthesia - but will most likely have a wonderful year of internal medicine to get through first. I think that IM is the perfect ground work for anesthesiology - its just the 3 hour evidence based discussions on whether 25 mg of drug X should be given BID or 50 mg QD. Arrrrgh.

 

[DrNick2006]

I see where there could be a confounder in my logic, lets explore. The statement was that if the applicant ranks the BI first then they have a 50% chance of matching there. When you rank a program first the match looks at that program's rank order list to see if you are on there. IF you are on there list anywhere (could be the 250th person) then that is where you match UNLESS they have already filled all there slots with others whom they ranked higher. So, we can correct for this by not only having a term RR for your chance of making thier rank order list, but a new term RM for your chance of matching there if you are on thier rank order list, based on how many people ahead of you also rank that program first, and where you think you are on their rank order list in comparison to available slots. The new equation is:

1-[(1-RR1)(RM1)*(1-RR2)(RM2)... (1-RRn)(RMn)]


Also, after the first postition on the rank order list I am unsure how the algorythm works for assigning a match. Wow, these equations are getting complicated now, I think I will have to sit down tonight to work out the exact equations ( I should be studying for boards) I think the thing to remember is that for all but the very 4 or five best programs, it is fair to assume that not everyone ahead of you has ranked that program first and RM is very close to 1.

I implicitly took this into acount under consideration #2 when considering RR, but now I have defined RR simply as your chance of being ranked and RM is now you postition within the rank list compared to others, and the relative ranking of that program by those ahead of you. I actually have to go do work, so I think instead of 5 places, everyone should rank 7 for now and I'll come up with a more exact answer later. I'm sure I can quanititate RM better given some thought, more to come.

 

[medlaw06]

I see where there could be a confounder in my logic, lets explore. The statement was that if the applicant ranks the BI first then they have a 50% chance of matching there. When you rank a program first the match looks at that program's rank order list to see if you are on there. IF you are on there list anywhere (could be the 250th person) then that is where you match UNLESS they have already filled all there slots with others whom they ranked higher. So, we can correct for this by not only having a term RR for your chance of making thier rank order list, but a new term RM for your chance of matching there if you are on thier rank order list, based on how many people ahead of you also rank that program first, and where you think you are on their rank order list in comparison to available slots. The new equation is:

1-[(1-RR1)(RM1)*(1-RR2)(RM2)... (1-RRn)(RMn)]


Also, after the first postition on the rank order list I am unsure how the algorythm works for assigning a match. Wow, these equations are getting complicated now, I think I will have to sit down tonight to work out the exact equations ( I should be studying for boards) I think the thing to remember is that for all but the very 4 or five best programs, it is fair to assume that not everyone ahead of you has ranked that program first and RM is very close to 1.

I implicitly took this into acount under consideration #2 when considering RR, but now I have defined RR simply as your chance of being ranked and RM is now you postition within the rank list compared to others, and the relative ranking of that program by those ahead of you. I actually have to go do work, so I think instead of 5 places, everyone should rank 7 for now and I'll come up with a more exact answer later. I'm sure I can quanititate RM better given some thought, more to come.

dude[ette]....you are SO making this stuff up.....I think.....

in all seriousness, I don't think that there CAN be a way to mathematically figure this out as someone else previously mentioned....take into consideration my fiance who interviewed @ B&W last year for anesthesiology...they interview around 100 for their 23 spots....and they stated that they were gonna rank her....now, she could have been ranked @ 24 and still not match IF the first 23 ALL had B&W as their #1..OR she would have matched there had ONLY 1 person matched elsewhere (assuming she is still # 24)...

the main point is is that you, the applicant, need to know WHERE EXACTLY YOU are being ranked by the program itself, which is obviously NOT going to be known....the reaosn why I say that is because if my fiance had known she was # 24, she may have only ranked 3 placed knowing that her chances of getting into B&W was high, but she wqould have ranked MORE places if she knew that she was #99 for B7W, #99 for Columbia, #99 for JHU, etc etc....

So the point is is that you can never know how many porgrams to rank since 1) you do not know WHERE you stand in the porgram's rank order list, and 2) you may not know how many applicants/interviewees are being interviewed, 3) you may not know how many applicants/interviewees are being ranked by that particular porgram...

for example, if porgram X interviews only 50 applicants for 30 spots and ranks all 50, then your chances at that place are higher than another program which interviews 500 for 30 spots and ranks 250....

since you, the applicant, do not know EXACTLY WHERE YOU STAND in that program's ROL, there is NO WAY you can know your likelihood of matching since IT ALSO DEPENDS ON WHAT OTHER APPLICANTS RANK THAT PROGRAM (see above example of finace and 1 person deciding NOT to match @ B&W)

Just my $0.000000002 since I am NOT a math person....I'm not saying that you are incorrect, but this is just another opinions.....

In any case, STRONG WORK!!!

 

[aredoubleyou]

dude[ette]....you are SO making this stuff up.....I think.....

in all seriousness, I don't think that there CAN be a way to mathematically figure this out as someone else previously mentioned....take into consideration my fiance who interviewed @ B&W last year for anesthesiology...they interview around 100 for their 23 spots....and they stated that they were gonna rank her....now, she could have been ranked @ 24 and still not match IF the first 23 ALL had B&W as their #1..OR she would have matched there had ONLY 1 person matched elsewhere (assuming she is still # 24)...

the main point is is that you, the applicant, need to know WHERE EXACTLY YOU are being ranked by the program itself, which is obviously NOT going to be known....the reaosn why I say that is because if my fiance had known she was # 24, she may have only ranked 3 placed knowing that her chances of getting into B&W was high, but she wqould have ranked MORE places if she knew that she was #99 for B7W, #99 for Columbia, #99 for JHU, etc etc....

So the point is is that you can never know how many porgrams to rank since 1) you do not know WHERE you stand in the porgram's rank order list, and 2) you may not know how many applicants/interviewees are being interviewed, 3) you may not know how many applicants/interviewees are being ranked by that particular porgram...

for example, if porgram X interviews only 50 applicants for 30 spots and ranks all 50, then your chances at that place are higher than another program which interviews 500 for 30 spots and ranks 250....

since you, the applicant, do not know EXACTLY WHERE YOU STAND in that program's ROL, there is NO WAY you can know your likelihood of matching since IT ALSO DEPENDS ON WHAT OTHER APPLICANTS RANK THAT PROGRAM (see above example of finace and 1 person deciding NOT to match @ B&W)

Just my $0.000000002 since I am NOT a math person....I'm not saying that you are incorrect, but this is just another opinions.....

In any case, STRONG WORK!!!

I think the whole point is that its supposed to be a probability and not an actual, for sure, number need to match. If it is a probability, you can put in any numbers you want so long as you ASSUME that your estimations have a 95% of being correct. And we all know what happens when you assume... 🙂