Step 1 P/F: Decision

[NeuResurgens]

---

Members don't see this ad.

Their stated reasoning for showing scaled scores was so that testers (and med schools) could see how widely people had cleared the Pass threshold. There is absolutely no way that in 1992, they expected their median would become our fail threshold, or that getting an 82% versus 88% would drastically change your specialty and program options.

And I definitely don't think the net effect of Step 1 Mania was a reduction in student stress levels. It's been increasing per the faculty that have watched it. The unknown is always scary to the first cohort (much like COVID for my year). But if we bump this thread in the year 2030, the current med students will no doubt laugh us off the forums for claiming our setup was the most fair or sensible or identified the best residents.

I just don't see how we can excuse them in 1992 like they're the victims. I think using a scored exam has pretty predictable consequences. In my opinion the increasing student stress is more likely to be attributed to the increasingly competitive nature of residency applications - using step 1 as a benchmark is more a symptom of that than the disease itself, and even though I think p/f would have been the right decision eventually, changing that now without a plan for better objective measures (preferably something that is more longitudinal rather than dependent on a single test day) will temporarily increase student stress in the near future.

 

[efle]

I just don't see how we can excuse them in 1992 like they're the victims. I think using a scored exam has pretty predictable consequences. In my opinion the increasing student stress is more likely to be attributed to the increasingly competitive nature of residency applications - using step 1 as a benchmark is more a symptom of that than the disease itself, and even though I think p/f would have been the right decision eventually, changing that now without a plan for better objective measures (preferably something that is more longitudinal rather than dependent on a single test day) will temporarily increase student stress in the near future.

It's not like the scores were immediately used that way either, though. 15 years into it, the IQR for Ortho matches was 220-245 when 220 was the national median. To repeat that, an average student could apply Ortho in 2006 and be within the matched IQR. It really wasn't until Application Fever took off that PDs started using Step 1 to filter by necessity. Nobody ever looked at Step 1 and said "wow, what a great Residency Aptitude Test this is that nobody else noticed in the last decade"!

Rather they said "I cannot read 500 applications for 5 spots. I will hit this Sort button and use this to filter because I must and there are no alternatives."

They couldn't possibly have predicted that ERAS would evolve like this in the 2010s or that their test, two decades after creation, would start getting abused to the modern degree.

 

[NeuResurgens]

It's not like the scores were immediately used that way either, though. 15 years into it, the IQR for Ortho matches was 220-245 when 220 was the national median. To repeat that, an average student could apply Ortho in 2006 and be within the matched IQR. It really wasn't until Application Fever took off that PDs started using Step 1 to filter by necessity. Nobody ever looked at Step 1 and said "wow, what a great Residency Aptitude Test this is that nobody else noticed in the last decade"!

Rather they said "I cannot read 500 applications for 5 spots. I will hit this Sort button and use this to filter because I must and there are no alternatives."

They couldn't possibly have predicted that ERAS would evolve like this in the 2010s or that their test, two decades after creation, would start getting abused to the modern degree.

This I agree with. I think that capping the number of residency apps would have made a better effect immediately - as the shotgun approach to increasingly competitive residencies has led to PDs needing something to screen people easily. I don't think there's any single metric in existence that is a fair one to screen people out on its merits alone (which is why I think the p/f decision was a wrong decision for the right reason) - so limiting the number of apps so that PDs could more completely evaluate applicants would be a step in the right direction. Of course, the logistics of this kind of policy aren't necessarily as straightforward as they might seem on the surface.

 

[efle]

I think that capping the number of residency apps would have made a better effect immediately

Agree. Pass/Fail Step 1, Pass/Fail Step 2, and capped applications is the only stable arrangement that I can see working

 

[GastriqueGraffin]

What does capping applications mean? First come, first served? The ridiculousness of this decision to make step 1 p/f is so obvious. They could have changed it to allow people to retake and not show the lower score, or combine the score as part of a formula, or a bunch of other options. Fact that this thread is still alive and the thoughts people are coming with are more proof that this was a bad decision.

 

[dohopeful13]

What does capping applications mean? First come, first served? The ridiculousness of this decision to make step 1 p/f is so obvious. They could have changed it to allow people to retake and not show the lower score, or combine the score as part of a formula, or a bunch of other options. Fact that this thread is still alive and the thoughts people are coming with are more proof that this was a bad decision.

It means applicants would have too limit the number of programs they can apply to, rather than sending as many as possible

 

[GastriqueGraffin]

It means applicants would have too limit the number of programs they can apply to, rather than sending as many as possible

How would that work? That would totally leave a bunch of programs and applicants unmatched. Might as well get rid of match then.

 

[efle]

How would that work? That would totally leave a bunch of programs and applicants unmatched. Might as well get rid of match then.

It's actually backed by good data. You can find lots of curves in AAMC and NRMP materials showing the percent chance of matching versus number of programs ranked. Generally speaking, your odds linearly increase per program until hitting a high plateau (like 95% or higher). At that point, there is diminishing and eventually negligible returns for additional ranks. Most of the time you only need ~10 ranks to hit the diminishing zone.

So, the idea would be that instead of Joey the Surgical Specialty Applicant going into ERAS and applying to 100 programs to generate himself 10 interviews, and forcing PDs to filter by Step score, we could instead have all the surgical applicants limited to only a few dozen applications. The numbers of applicant and numbers of seats are the same, and the numbers of interviews handed out should be the same, but this would drastically reduce the application load on any randomly selected program. Ideally, this means the program can do much more thorough review and not rely on Step filters/preferences as a crutch. And mathematically speaking, as long as this is applied as a blanket rule to everyone, there shouldn't be any reduction in people's numbers of interviews or match rate. If anything, it would drastically increase your odds of interviewing at your actual handful of favorites, since they would be much less inundated with apps.

In some specialties like Plastics, it's become common advise to simply check off every single program in ERAS. That is absurd and likely represents what the Match would have trended towards in other competitive specialties. When all it takes is a mouse click and a small fee to throw more programs on your list, an escalating arms race of applying to as many programs as possible is the inevitable result.

 

[GastriqueGraffin]

It's actually backed by good data. You can find lots of curves in AAMC and NRMP materials showing the percent chance of matching versus number of programs ranked. Generally speaking, your odds linearly increase per program until hitting a high plateau (like 95% or higher). At that point, there is diminishing and eventually negligible returns for additional ranks. Most of the time you only need ~10 ranks to hit the diminishing zone.

So, the idea would be that instead of Joey the Surgical Specialty Applicant going into ERAS and applying to 100 programs to generate himself 10 interviews, and forcing PDs to filter by Step score, we could instead have all the surgical applicants limited to only a few dozen applications. The numbers of applicant and numbers of seats are the same, and the numbers of interviews handed out should be the same, but this would drastically reduce the application load on any randomly selected program. Ideally, this means the program can do much more thorough review and not rely on Step filters/preferences as a crutch. And mathematically speaking, as long as this is applied as a blanket rule to everyone, there shouldn't be any reduction in people's numbers of interviews or match rate. If anything, it would drastically increase your odds of interviewing at your actual handful of favorites, since they would be much less inundated with apps.

In some specialties like Plastics, it's become common advise to simply check off every single program in ERAS. That is absurd and likely represents what the Match would have trended towards in other competitive specialties. When all it takes is a mouse click and a small fee to throw more programs on your list, an escalating arms race of applying to as many programs as possible is the inevitable result.

Problem becomes when everyone tries to go to the big city/big name places, thinking that they are special because without step they have some fake research and other extracurriculars. Nobody applies to the community, middle of nowhere program or less competitive specialty. Match day comes and now we see many mid and low tier programs unfilled and many applicants without a spot scrambling to find something. You are right in that ranking beyond a certain number doesn’t help (usually around 10 programs) but in order to rank them, you have to be invited to interview with them.

 

[efle]

Problem becomes when everyone tries to go to the big city/big name places, thinking that they are special because without step they have some fake research and other extracurriculars. Nobody applies to the community, middle of nowhere program or less competitive specialty. Match day comes and now we see many mid and low tier programs unfilled and many applicants without a spot scrambling to find something. You are right in that ranking beyond a certain number doesn’t help (usually around 10 programs) but in order to rank them, you have to be invited to interview with them.

That just means the SOAP gets bigger in the first few years. But I don't even think that would happen. I think the pattern would mirror medical school admissions (or residency applications of a couple decades ago) where you apply to a broad, mixed list of a handful of top-tier dream spots, a handful of well matched mid tiers you are competitive for and would fit with, and then a handful of "safety" where you are more highly competitive.

It's a zero sum game with the same numbers of applicants and seats regardless of whether a cap is present. But a cap protects the PDs from having to each review 100 apps per spot.

 

[GastriqueGraffin]

That just means the SOAP gets bigger in the first few years. But I don't even think that would happen. I think the pattern would mirror medical school admissions (or residency applications of a couple decades ago) where you apply to a broad, mixed list of a handful of top-tier dream spots, a handful of well matched mid tiers you are competitive for and would fit with, and then a handful of "safety" where you are more highly competitive.

It's a zero sum game with the same numbers of applicants and seats regardless of whether a cap is present. But a cap protects the PDs from having to each review 100 apps per spot.

People apply very broadly to medical schools, especially if coming from competitive states. But there is no match so they see early on where they stand.
I honestly don’t envy PDs once this comes into effect. Step 1 was a very good marker of an applicants ability to study and pass the boards. It shouldn’t be the only factor because clinical rotations usually reflect some studying and some bedside manner (mixed with luck and baseline personality of the student and grader). Step 1 was the main reason anyone ever studied preclinical stuff. I hope after programs see how bad of a decision this was, it gets undone in a better way like what I mentioned before (creating some kind of overall formula to generate a score for an applicant).

 

[slowthai]

It's actually backed by good data. You can find lots of curves in AAMC and NRMP materials showing the percent chance of matching versus number of programs ranked. Generally speaking, your odds linearly increase per program until hitting a high plateau (like 95% or higher). At that point, there is diminishing and eventually negligible returns for additional ranks. Most of the time you only need ~10 ranks to hit the diminishing zone.

So, the idea would be that instead of Joey the Surgical Specialty Applicant going into ERAS and applying to 100 programs to generate himself 10 interviews, and forcing PDs to filter by Step score, we could instead have all the surgical applicants limited to only a few dozen applications. The numbers of applicant and numbers of seats are the same, and the numbers of interviews handed out should be the same, but this would drastically reduce the application load on any randomly selected program. Ideally, this means the program can do much more thorough review and not rely on Step filters/preferences as a crutch. And mathematically speaking, as long as this is applied as a blanket rule to everyone, there shouldn't be any reduction in people's numbers of interviews or match rate. If anything, it would drastically increase your odds of interviewing at your actual handful of favorites, since they would be much less inundated with apps.

In some specialties like Plastics, it's become common advise to simply check off every single program in ERAS. That is absurd and likely represents what the Match would have trended towards in other competitive specialties. When all it takes is a mouse click and a small fee to throw more programs on your list, an escalating arms race of applying to as many programs as possible is the inevitable result.

Plus the top tier applicants won't be gobbling up all the invites and interviewing unnecessarily just to increase their IV count for bragging rights.

 

[fldoctorgirl]

It's not like the scores were immediately used that way either, though. 15 years into it, the IQR for Ortho matches was 220-245 when 220 was the national median. To repeat that, an average student could apply Ortho in 2006 and be within the matched IQR. It really wasn't until Application Fever took off that PDs started using Step 1 to filter by necessity. Nobody ever looked at Step 1 and said "wow, what a great Residency Aptitude Test this is that nobody else noticed in the last decade"!

Rather they said "I cannot read 500 applications for 5 spots. I will hit this Sort button and use this to filter because I must and there are no alternatives."

They couldn't possibly have predicted that ERAS would evolve like this in the 2010s or that their test, two decades after creation, would start getting abused to the modern degree.

This would also get rid of the game of PDs having to "yield protect" by thinking that an applicant only applied to their program as a safety. PDs would know that people applied to their program because they truly wanted to be there.

 

[GastriqueGraffin]

So basically everyone is saying it’s ok that the quality of applicant goes down, as long as it helps themselves

 

[Matthew9Thirtyfive]

So basically everyone is saying it’s ok that the quality of applicant goes down, as long as it helps themselves

Not sure how you got there.

 

[fldoctorgirl]

So basically everyone is saying it’s ok that the quality of applicant goes down, as long as it helps themselves

No. I agree that Step 1 should remain scored. I think the answer to the problem would've been limiting the amount of residencies people could apply to. Making Step 1 P/F does nothing, as programs will just use Step 2. Once that goes P/F, programs will use research or school prestige or something else. It just keeps putting a bandaid on the problem without actually fixing it.

And that's besides the point of the argument to be made for whether Step score correlates to quality of applicant, but that's another discussion.

 

[GastriqueGraffin]

Not sure how you got there.

Based on above comment saying that everyone would be limited in the number of places they can interview would decrease spots taken by top applicants.

 

[longhaul3]

So basically everyone is saying it’s ok that the quality of applicant goes down, as long as it helps themselves

What matters is the quality of the matched applicants, which would not change.

As it stands the quality of applicant is the same at every program because everyone applies to every program. The better applicants match at the better programs regardless. The weaker programs would not be missing out on top applicants because they were not going to match there anyway.

 

[GastriqueGraffin]

Not sure how you got there.

Based on above comment saying that everyone would be limited in the number of places they can interview would decrease spots taken by top applicants.

 

[efle]

Based on above comment saying that everyone would be limited in the number of places they can interview would decrease spots taken by top applicants.

Correct, but the quality of the Match isn't predicated on how many interviews are given to top applicants. It's predicated on how people match. They still take up 1 seat each.

If we can either have Joey taking 10 interviews and matching to Hopkins ENT, or 25 interviews and matching at Hopkins ENT, we should prefer the former, not the latter. The outcome is the same and only gain is to be had (from the other 15 interviews being better spent and from all programs having to read less applications when choosing how to spend them).

To give a more real world example, we had an applicant from my school that received >40 interview invitations and ended up home matching. That is an absurd amount of waste. The answer is caps. That's an extreme case obviously, but illustrates my point.

 

[NeuResurgens]

That just means the SOAP gets bigger in the first few years. But I don't even think that would happen. I think the pattern would mirror medical school admissions (or residency applications of a couple decades ago) where you apply to a broad, mixed list of a handful of top-tier dream spots, a handful of well matched mid tiers you are competitive for and would fit with, and then a handful of "safety" where you are more highly competitive.

It's a zero sum game with the same numbers of applicants and seats regardless of whether a cap is present. But a cap protects the PDs from having to each review 100 apps per spot.

Exactly. It'll force applicants to be more judicious in where they apply. Even the most competitive specialties still have a MEAN step 1 score of 245-248, meaning about half of all applicants fall below that. There are still a lot of people applying to these specialties that wouldn't necessarily be shoe-ins for the top programs, and thus both the applicants and the top programs don't really need to waste either of their time.

And, even if SOAP gets bigger - didn't we just say we'd be left with a bunch of unfilled low-tier programs in competitive specialties, and lots of unmatched applicants to those specialties? There you go - we literally have a mechanism of fixing that in every match cycle.

 

[GastriqueGraffin]

What is the point of all this? How does this improve quality of an applicant? The problem with the decision to make step 1 p/f is that it doesn’t do that at all.

 

[Chibucks15]

What is the point of all this? How does this improve quality of an applicant? The problem with the decision to make step 1 p/f is that it doesn’t do that at all.

It wasn't mean to improve the quality of the applicant. Step 1 is not a great indicator of quality of an applicant by any means, it just means you can study and have good test taking ability. It was meant to lessen the stress that was literally killing some students because their whole futures ride on this test.

Making it P/F was how it was always meant to be utilized anyway. However, there are a lot of ways that the system needs fixing regardless. But see any and all of the 10000000 pages of comments on this

 

[Matthew9Thirtyfive]

Based on above comment saying that everyone would be limited in the number of places they can interview would decrease spots taken by top applicants.

Right. Someone who is MGH material likely isn't matching at Decent Community Program unless they really have a reason to go there. There are cases on this forum just from this year of people with great apps who didn't match most likely because the less prestigious programs on their rank list assumed they'd be going somewhere better (and IIRC, someone had that confirmed by a program). Limiting apps would help everyone because it would force people to really critically look at their app to see how competitive they are AND really think about which programs they want to apply to. The programs would know that if someone applied there, they are actually interested because they probably wouldn't waste one of their limited apps on a throwaway program.

It doesn't change the quality of the residents.

 

[GastriqueGraffin]

It wasn't mean to improve the quality of the applicant. Step 1 is not a great indicator of quality of an applicant by any means, it just means you can study and have good test taking ability. It was meant to lessen the stress that was literally killing some students because their whole futures ride on this test.

Making it P/F was how it was always meant to be utilized anyway. However, there are a lot of ways that the system needs fixing regardless. But see any and all of the 10000000 pages of comments on this

This is the same bs that everyone says about any standardized test (sat, act, mcat, etc). Why not get rid of all of them? Because everyone knows that there is a correlation with someone’s ability to study for a test and how studious they are. Furthermore, that expression of “I learned that in medical school” when talking about a zebra case largely comes from the preclinical and step 1 days. As does the understanding how the biochemistry, physiology, pharmacology, pathology of diseases. Step 1 forced (good) students to get a much better grasp of these. Memorizing clinical guidelines or learning how to do a procedure can be done by pretty much anyone but understanding of the how and why requires preclinical knowledge.

I agree that it was high pressure and they could have fixed it by either giving people hidden attempts and using only the best score or creating a formula in which step 1 score was just a part of it.

 

[Chibucks15]

This is the same bs that everyone says about any standardized test (sat, act, mcat, etc). Why not get rid of all of them? Because everyone knows that there is a correlation with someone’s ability to study for a test and how studious they are. Furthermore, that expression of “I learned that in medical school” when talking about a zebra case largely comes from the preclinical and step 1 days. As does the understanding how the biochemistry, physiology, pharmacology, pathology of diseases. Step 1 forced (good) students to get a much better grasp of these. Memorizing clinical guidelines or learning how to do a procedure can be done by pretty much anyone but understanding of the how and why requires preclinical knowledge.

I agree that it was high pressure and they could have fixed it by either giving people hidden attempts and using only the best score or creating a formula in which step 1 score was just a part of it.

I’ve forgotten the majority of the nonsense we learned on step 1 and I did fine on it. I’ve had great clinical evals too. it doesn’t mean jack towards quality of student or future resident. Step 1 doesn’t show much of anything besides self worth on SDN and an easy way to stratify applications because every applicant will have a step 1 and not every applicant will have a step 2 by application time.

This has been discussed like crazy a million times it isn’t worth rehashing

 

[slowthai]

So basically everyone is saying it’s ok that the quality of applicant goes down, as long as it helps themselves

Why should people that would be just as capable residents be shut out of a specialty just because a T5 superstar went on 20+ interviews? It doesn't make sense. They can only take one spot at one residency. Limiting apps redistributes interviews to the rest of the applicants, including the mid tier and the low tier.

Furthermore, it helps even the top tier candidate because they can save money, time, and energy by not flying out to these extra interviews.

 

[Matthew9Thirtyfive]

I’ve forgotten the majority of the nonsense we learned on step 1 and I did fine on it. I’ve had great clinical evals too. it doesn’t mean jack towards quality of student or future resident. Step 1 doesn’t show much of anything besides self worth on SDN and an easy way to stratify applications because every applicant will have a step 1 and not every applicant will have a step 2 by application time.

This has been discussed like crazy a million times it isn’t worth rehashing

There was a study in the American Journal of Surgery that showed residents with a higher step 1 score did better on general surgery boards and were more likely to get better evals during residency. But the authors admitted that only 50% of the residents with high step scores had superior evals and that you can do well on boards without a killer step 1 score. Their suggestion was to use step 1 as part of evaluating residents, but to not rely too heavily on it.

 

[Matthew9Thirtyfive]

Limiting apps redistributes interviews to the rest of the applicants,

lol jk

 

[Chibucks15]

There was a study in the American Journal of Surgery that showed residents with a higher step 1 score did better on general surgery boards and were more likely to get better evals during residency. But the authors admitted that only 50% of the residents with high step scores had superior evals and that you can do well on boards without a killer step 1 score. Their suggestion was to use step 1 as part of evaluating residents, but to not rely too heavily on it.

Eh to me, that’s exactly what my point is. Use it as a whole review but it really isn’t as important as SDN makes it out to be and it going P/F will not change the quality of residents very much at all

 

[Matthew9Thirtyfive]

Eh to me, that’s exactly what my point is. Use it as a whole review but it really isn’t as important as SDN makes it out to be and it going P/F will not change the quality of residents very much at all

Yeah, I was saying that there is research that agrees with you.

 

[Chibucks15]

Yeah, I was saying that there is research that agrees with you.

So used to pushback On step 1 crap I read it wrong haha my bad

 

[GastriqueGraffin]

Why should people that would be just as capable residents be shut out of a specialty just because a T5 superstar went on 20+ interviews? It doesn't make sense. They can only take one spot at one residency. Limiting apps redistributes interviews to the rest of the applicants, including the mid tier and the low tier.

Furthermore, it helps even the top tier candidate because they can save money, time, and energy flying out to these extra interviews.

If that was true then these specialties would be unfilled at the end. When was the last time derm went unfulfilled for example? It’s just a matter of how will they be able to distinguish who is a good applicant on paper. There is no good replacement for preclinical knowledge evaluation than a standardized exam. I do understand people with lower scores pretending like they agree with p/f because it’s self-serving. But it’s not sincere in most cases.

 

[GastriqueGraffin]

View attachment 299476


lol jk

Funny ‘cause it’s true

 

[AnatomyGrey12]

I’ve forgotten the majority of the nonsense we learned on step 1 and I did fine on it. I’ve had great clinical evals too. it doesn’t mean jack towards quality of student or future resident. Step 1 doesn’t show much of anything besides self worth on SDN and an easy way to stratify applications because every applicant will have a step 1 and not every applicant will have a step 2 by application time.

This has been discussed like crazy a million times it isn’t worth rehashing

This argument assumes that we have something else that actually does this. We don't. Literally none of the current metrics we have predicts who is going to be a good resident or not. Step 2 has the same statistical issues as Step 1, clinical grades are a complete crapshoot, and research is almost all poop.

We shouldn't have gotten rid of Step 1 without a plan. The NBME clearly didn't give this any foresight, as evidenced by their own statements on the matter.

For the record, I do agree with residency application caps. That would have gone much farther to help the problem than changing Step 1 to P/F. Step 1 mania was a symptom, not the disease itself.

 

[Chibucks15]

If that was true then these specialties would be unfilled at the end. When was the last time derm went unfulfilled for example? It’s just a matter of how will they be able to distinguish who is a good applicant on paper. There is no good replacement for preclinical knowledge evaluation than a standardized exam. I do understand people with lower scores pretending like they agree with p/f because it’s self-serving. But it’s not sincere in most cases.

It’s also very self serving for people with high scores to be all up in arms about it. It goes both ways. The reality is that preclinical knowledge is by in large not a good predictor of capability as a physician. Yes it’s important for the training of a physician, but there are much better ways to actually stratify future physicians then a test on PhD material. As mentioned above. But I’m not gonna get into it any more because it’s been beaten to death. It’s P/F now and that’s how it is. Adapt and overcome

 

[slowthai]

If that was true then these specialties would be unfilled at the end. When was the last time derm went unfulfilled for example? It’s just a matter of how will they be able to distinguish who is a good applicant on paper. There is no good replacement for preclinical knowledge evaluation than a standardized exam. I do understand people with lower scores pretending like they agree with p/f because it’s self-serving. But it’s not sincere in most cases.

No, it just means that there's less heterogeneity of applicants in these fields currently.

And like I've said before, I completely disagree with step going pass/fail. It doesn't fix anything, it just makes the "mania" worse, but it'll be for step 2 instead. It was a total mistake.

But that's neither here nor there as far as the real reasons for this change. It's a 100% self-serving decision for the NBME. They'll make more money out of this and they'll get those annoying administrators off their backs.

 

[Chibucks15]

This argument assumes that we have something else that actually does this. We don't. Literally none of the current metrics we have predicts who is going to be a good resident or not. Step 2 has the same statistical issues as Step 1, clinical grades are a complete crapshoot, and research is almost all poop.

We shouldn't have gotten rid of Step 1 without a plan. The NBME clearly didn't give this any foresight, as evidenced by their own statements on the matter.

For the record, I do agree with residency application caps. That would have gone much farther to help the problem than changing Step 1 to P/F. Step 1 mania was a symptom, not the disease itself.

Definitely agree. I just am speaking to the fact that Step 1 is largely clinically irrelevant, and should not be taken to the extent that it has been. Application caps, connection to area/program, and board scores/letters viewed as a entire application definitely is the better plan moving forward. It just doesn’t seem like any admins ever really have a plan haha

 

[ace_inhibitor111]

This discussion doesn't take into account why students are blanket applying in the first place. We only have limited information on how "competitive" we are and many programs keep us in the dark in terms of what they look for in applicants. It's not like I can read my LORs, and I have no idea how to compare my publications and clinical grades when I don't know what people in my class have and pubs/posters/presentations are lumped into one category. Pass/fail Step 1 will only worsen the problem because now students have even less information on how competitive they are.

Many program directors have geographic preferences or medical school preferences. Some don't read the personal statement, some judge it highly. The main people who would benefit from limited apps are those at Top 20 or with AOA, who already know their best chances are those at similarly "high-ranking" institutions.

 

[fldoctorgirl]

The main people who would benefit from limited apps are those at Top 20 or with AOA, who already know their best chances are those at similarly "high-ranking" institutions.

But the end result is the same: these people are matching at those high-ranking institutions regardless. However, in the current situation they're going on 20+ interviews in some cases, when they know they don't want to go to any of the lower-ranked/community programs. If they were barred from applying to a million residencies, these spots would open up to others.

 

[ace_inhibitor111]

But the end result is the same: these people are matching at those high-ranking institutions regardless. However, in the current situation they're going on 20+ interviews in some cases, when they know they don't want to go to any of the lower-ranked/community programs. If they were barred from applying to a million residencies, these spots would open up to others.

Very few people at Top 20 or with AOA are applying to low-rank programs for the hell of it. The only reason they would actually do that is if they were interested in the location. Most people shooting out tons of apps are those at low-rank schools or who don't know how competitive they are. DO students applied to 17 more apps on average than MDs this year.

Take a look at these statistics from Harvard. Students are applying to half the average of apps for each specialty for USGs because they know they can. Most students don't have that privilege.

Log In - Harvard Medical School

collaborate.hms.harvard.edu

 

[fldoctorgirl]

Very few people at Top 20 or with AOA are applying to low-rank programs for the hell of it. The only reason they would actually do that is if they were interested in the location. Most people shooting out tons of apps are those at low-rank schools or who don't know how competitive they are. DO students applied to 17 more apps on average than MDs this year.

Take a look at these statistics from Harvard. Students are applying to half the average of apps for each specialty for USGs because they know they can. Most students don't have that privilege.

Log In - Harvard Medical School

collaborate.hms.harvard.edu

There was a thread on this forum a few days ago about a high-stat applicant who should've not only matched, but matched insanely well, who ended up having to SOAP. A PD at one of the programs she interviewed at literally said that they didn't rank her highly because they were sure she would go to a better program. My point is, if we limit the amount of apps, people can only send apps to places they are truly interested in going (for whatever reasons they may personally have). There's no doubt that PDs play yield protection and limiting apps would fix that.

 

[efle]

This is the same bs that everyone says about any standardized test (sat, act, mcat, etc). Why not get rid of all of them? Because everyone knows that there is a correlation with someone’s ability to study for a test and how studious they are. Furthermore, that expression of “I learned that in medical school” when talking about a zebra case largely comes from the preclinical and step 1 days. As does the understanding how the biochemistry, physiology, pharmacology, pathology of diseases. Step 1 forced (good) students to get a much better grasp of these. Memorizing clinical guidelines or learning how to do a procedure can be done by pretty much anyone but understanding of the how and why requires preclinical knowledge.

I agree that it was high pressure and they could have fixed it by either giving people hidden attempts and using only the best score or creating a formula in which step 1 score was just a part of it.

The attendings training and teaching you on the wards overwhelmingly scored in the 200s-210s or lower, if they took it at all. Yet we have no problem respecting them, or at least, I don't. The suggestion that the existence of the test was the important part, not the content itself, just underscores how badly we needed this change. Having everyone memorize digits of Pi to find who is the most studious should feel like an effective argument ad absurdum, not like an actual good idea.

Very few people at Top 20 or with AOA are applying to low-rank programs for the hell of it. The only reason they would actually do that is if they were interested in the location. Most people shooting out tons of apps are those at low-rank schools or who don't know how competitive they are. DO students applied to 17 more apps on average than MDs this year.

Take a look at these statistics from Harvard. Students are applying to half the average of apps for each specialty for USGs because they know they can. Most students don't have that privilege.

Log In - Harvard Medical School

collaborate.hms.harvard.edu

I'll add that it's not just a maldistribution of applications, but also of who gets interviewed. A shocking fraction of interviews gets absorbed by a small handful of applicants. In the worst cases like Family Medicine and Internal Medicine, a full 50% of interviews are being taken by only 7-12% of applicants.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6883635/pdf/nihms-1055134.pdf

The opportunity cost of this inequity is huge. So, so many interviews are getting wasted on a handful of superstars that are all likely to land in their first few ranks.

 

[longhaul3]

Why should people that would be just as capable residents be shut out of a specialty just because a T5 superstar went on 20+ interviews? It doesn't make sense. They can only take one spot at one residency. Limiting apps redistributes interviews to the rest of the applicants, including the mid tier and the low tier.

Furthermore, it helps even the top tier candidate because they can save money, time, and energy by not flying out to these extra interviews.

This doesn't add up—if top applicants are taking up all the interview slots and matching at only a few programs, you would expect the lesser programs to go unfilled, which doesn't happen in competitive specialties. The point of restricting applications is that the same number of people will match to the same number of slots with fewer interviews each. Over-application is not the reason that some people get shut out of competitive specialties.

 

[efle]

This doesn't add up—if top applicants are taking up all the interview slots and matching at only a few programs, you would expect the lesser programs to go unfilled, which doesn't happen in competitive specialties. The point of restricting applications is that the same number of people will match to the same number of slots with fewer interviews each. Over-application is not the reason that some people get shut out of competitive specialties.

It does add up, in the sense that the quality of their average cohort would be better. Right now if you're a middling program it's smart to be playing defensive. You have to either 1) interview an absurd number of superstars to fill a single slot at your program or 2) yield protect against strong apps because it's not safe to trust they'll rank you well.

You can safely protect yourself from having to SOAP, but only by either way over-inviting and hoping to get lucky or by intentionally hamstringing the quality of your interviewees to ensure you fill.

Imagine, instead, that you knew every single application to your program was seriously considering you because they only have a few precious applications to send. Boom, now you can safely interview all your highly desired applicants and not have to play the yield game. You still fill your program, but now you do it with less apps to review, less wasted interviews and more certainty of applicant interest.

 

[AnatomyGrey12]

It does add up, in the sense that the quality of their average cohort would be better. Right now if you're a middling program it's smart to be playing defensive. You have to either 1) interview an absurd number of superstars to fill a single slot at your program or 2) yield protect against strong apps because it's not safe to trust they'll rank you well.

You can safely protect yourself from having to SOAP, but only by either way over-inviting and hoping to get lucky or by intentionally hamstringing the quality of your interviewees to ensure you fill.

Imagine, instead, that you knew every single application to your program was seriously considering you because they only have a few precious applications to send. Boom, now you can safely interview all your highly desired applicants and not have to play the yield game. You still fill your program, but now you do it with less apps to review, less wasted interviews and more certainty of applicant interest.

Out of curiosity what would you say a good cap would be? I'm thinking around 50.

 

[Chibucks15]

Out of curiosity what would you say a good cap would be? I'm thinking around 50.

I think it would have to depend on specialty. Like 1/3 of total number of programs or something like that

 

[ace_inhibitor111]

I'll add that it's not just a maldistribution of applications, but also of who gets interviewed. A shocking fraction of interviews gets absorbed by a small handful of applicants. In the worst cases like Family Medicine and Internal Medicine, a full 50% of interviews are being taken by only 7-12% of applicants.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6883635/pdf/nihms-1055134.pdf

The opportunity cost of this inequity is huge. So, so many interviews are getting wasted on a handful of superstars that are all likely to land in their first few ranks.

Hmmm... That's interesting because from what I'm getting from the graph is that there is GREATER interview equality with more competitive specialties. These same specialties have applicants applying to over half the programs.

 

[efle]

Out of curiosity what would you say a good cap would be? I'm thinking around 50.

Maybe peg it to the specialty? Some of the outliers need a lot more apps/interviews to reach a safe plateau. But yeah capping at 50 would at least big a big improvement over the >80 that is becoming normal in some competitive situations.

 

[efle]

Hmmm... That's interesting because from what I'm getting from the graph is that there is GREATER interview equality with more competitive specialties. These same specialties have applicants applying to over half the programs.

Agree that trend is less present in the surgical fields, but that's probably because the range of quality is already self selected to be "great to amazing", rather than a wider range in something like IM or general surgery where a 250+/AOA is relatively much more rare.