New Token System for ENT Interview Invites

[slowthai]

Preference Signaling: Winners and Losers Edition

On August 19, 2020, the Otolaryngology Program Directors Organization (OPDO) announced an unprecedented change to the residency selection process. Let’s review what happened – and then …

thesheriffofsodium.com

Preliminary thoughts:

This is a hundred million times better than the pathetic love letter gamesmanship that people are doing now.

I think the tokens should be cut down to 3 maximum. I think that having a top 3 will result in applicants not being penalized as severely because you can only show interest in a limited number of programs. This is just my guess, though. We'll see how everything plays out during this upcoming interview season. All the other specialties should hurry up and follow suit.

---

Members don't see this ad.

 

[Frogger27]

better idea than secondary apps like they did a few years ago lol

 

[slowthai]

better idea than secondary apps like they did a few years ago lol

Exactly, lol

 

[Matthew9Thirtyfive]

Seems like a reasonable idea.

 

[IMGIMG MD]

Does it hurt IMG chances of matching?

 

[slowthai]

Does it hurt IMG chances of matching?

I don't see how it could. I mean, you would actually be able to show real, objective interest in the places that have a history of taking IMGs, potentially helping your chances.

 

[slowthai]

@operaman @Wordead What do you think?

 

[operaman]

I think this is a good idea but possibly insufficient for the current dilemma. The all zoom interview thing is a big wild card and I wonder if removing geographic and time limits will allow applicants to do 25-30 interviews rather than the more typical 10-15. Hopefully programs interview more people to compensate or else everyone is going to be interviewing the same group of 50-60 applicants for 315 spots nationwide!

So it’s into this context that we have the 5 internet flag thing. Personally I think it’s a great idea. We struggled in my residency to gauge from applications who was really interested in coming. We were one of the top programs but even we had to be careful. There were times we interviewed a lot of people who were unlikely to rank us highly based on geography and other factors, and those years we would drop farther down our list. If we had a way for people to signal interest when their CV suggests otherwise would be very helpful.

In the past students could do aways to show interest or counteract a regional bias, but that’s gone this season. Hopefully this helps that.

For us and many programs, you end up with a short list from which interviews are offered. The interest flags will probably be very helpful when making decisions at this point.

 

[Wordead]

@operaman @Wordead What do you think?

I don't know I'm not a fan I feel it adds another level of strategy and gamesmanship but I understand that something had to be done.

If I were an applicant I'd be agonizing - do I signal the more desirable programs and potentially "waste" a token? Or signal all "lower level" programs to maximize my odds of matching?

 

[slowthai]

I don't know I'm not a fan I feel it adds another level of strategy and gamesmanship but I understand that something had to be done.

If I were an applicant I'd be agonizing - do I signal the more desirable programs and potentially "waste" a token? Or signal all "lower level" programs to maximize my odds of matching?

Just freeballing here, but I think that the strategy would have to be tailored to the applicant based on overall competitiveness and geographic preferences.

If you're a superstar, your tokens would hold serious weight at the least competitive programs because they'd know you're not BS'ing when you spend a token on them and they wouldn't rank you low like these programs usually do. If you're an uncompetitive applicant, you'd better spend all your coins at places you have a realistic chance at. If you're middle of the road, you could spend tokens on where you truly want to go and maybe have a few "safeties". If you absolutely need/want to be in a certain area, you'd obviously spend all your coins on programs within a given region, of course taking your competitiveness into consideration.

 

[slowthai]

What's also interesting is that one of the new "rules" says that programs aren't allowed to ask applicants where they spent their tokens. That's like telling a 2 year old not to do something. It's guaran-freakin-teed

 

[Iz_FtB]

FEWER applicants are taking a research year this cycle and opting to graduate in 4 years, meaning more potential applicants.

On top of that, there is the obvious swing toward each applicant planning on attending more interviews.

SAGE Journals: Your gateway to world-class research journals

Subscription and open access journals from SAGE Publishing, the world's leading independent academic publisher.

doi.org

Gonna be a funny cycle.

 

[TheVagüs]

Just wanted to chime in regarding the interview counts possibly getting out of control. In ophthalmology our powers that be have capped the interview count at 20. Plenty of room to ensure a strong match, and it should help taper interview inflation this cycle. Perhaps ENT will do the same.

 

[slowthai]

Just wanted to chime in regarding the interview counts possibly getting out of control. In ophthalmology our powers that be have capped the interview count at 20. Plenty of room to ensure a strong match, and it should help taper interview inflation this cycle. Perhaps ENT will do the same.

Man, ophtho has just been leading the way this year. First a move to integrated and now this? Color me impressed

 

[efle]

The cynic in me sees this as Early Decision Metagame: Residency Edition. In order to become as selective as possible, most highly ranked colleges have started filling ~50% or more of their class using Early Decision applicants. Hope this doesn't become the same story

 

[Fatalis]

Does it hurt IMG chances of matching?

IMGs don’t generally match into ENT unless the institution knows them very well, etc. so no... it won’t hurt there chances lol

 

[operaman]

I think it’s key to point out that this is aimed at the pre interview stage only.

Sure there would be some gamesmanship, but no more than we already have in picking aways and whatnot. I don’t think anyone cares where else you used your tokens because we get that applicants are interested in many programs.

The big question that will surely be the topic of a paper in a few years will be whether these flags actually mean anything. Are they just a game piece or do they actually signal an increased chance of matching that applicant?

Interviews are traditionally such valuable slots - we would interview only 40 people out of ~450 applicants, so giving one to someone who would much rather match on the opposite coast robs us of a chance to meet a similarly qualified applicant who would much prefer to train with us. I expressed some very specific career plans and research interests in my personal statement and got pretty pan rejected by programs that had little to offer in those areas. They were savvy enough to know I would be ranking them near the bottom of my list anyhow so why waste a slot on me even though I was a very competitive applicant.

Every interview comes with the opportunity cost of the applicant you didn’t meet. No program likes to drop way down their rank list. You still get great people because our applicant pool is amazing, but if you drop to 30th/40 to fill a class of 3-5, you do wonder why 20+ people decided to train elsewhere. You think about the 60+ people on your short list who looked equally good on paper that you might have interviewed instead.

As such, I think the flags will have minimal impact on actual rank list order. Once you’ve burned an interview slot on someone it’s gone. Getting the best people with the best fit far outweighs how far you fall down the list.

 

[mehc012]

I think this is a good idea but possibly insufficient for the current dilemma. The all zoom interview thing is a big wild card and I wonder if removing geographic and time limits will allow applicants to do 25-30 interviews rather than the more typical 10-15. Hopefully programs interview more people to compensate or else everyone is going to be interviewing the same group of 50-60 applicants for 315 spots nationwide!

So it’s into this context that we have the 5 internet flag thing. Personally I think it’s a great idea. We struggled in my residency to gauge from applications who was really interested in coming. We were one of the top programs but even we had to be careful. There were times we interviewed a lot of people who were unlikely to rank us highly based on geography and other factors, and those years we would drop farther down our list. If we had a way for people to signal interest when their CV suggests otherwise would be very helpful.

In the past students could do aways to show interest or counteract a regional bias, but that’s gone this season. Hopefully this helps that.

For us and many programs, you end up with a short list from which interviews are offered. The interest flags will probably be very helpful when making decisions at this point.

Alternatively, they could just stop using "how far down a program goes on its rank list" as a status/prestige symbol and just let the Match algorithm work the way it's supposed to work. Then there'd be no need to mark people down or up based on the amount of interest they demonstrate. The entire Match ranking algorithm thing is ALREADY set up to take mutual interest into account when placing applicants...once PDs start trying to out-think the system they shank the whole damn thing.

Instead of tokens, just make it that PDs aren't allowed to share how far down the rank list they went, period. Much easier solution.

No program likes to drop way down their rank list. You still get great people because our applicant pool is amazing, but if you drop to 30th/40 to fill a class of 3-5, you do wonder why 20+ people decided to train elsewhere.

Sorry, but I can't really feel badly for a program getting 5 great incoming interns but their ego being a bit bruised in comparison to an applicant who would have been an excellent resident there and that the program would have loved to have getting shafted and changing their future career path (or potentially even not Matching) because PDs are trying to play a guessing game when there is literally a system ALREADY in place to take the guesswork out of it. Programs dipping to slot 30 to get the best shot at the best possible intern class they can have, while simultaneously NOT screwing applicants, sounds like a win-win to me.

We don't need a new solution. We need to actually USE the existing one.

 

[slowthai]

Alternatively, they could just stop using "how far down a program goes on its rank list" as a status/prestige symbol and just let the Match algorithm work the way it's supposed to work. Then there'd be no need to mark people down or up based on the amount of interest they demonstrate. The entire Match ranking algorithm thing is ALREADY set up to take mutual interest into account when placing applicants...once PDs start trying to out-think the system they shank the whole damn thing.

Instead of tokens, just make it that PDs aren't allowed to share how far down the rank list they went, period. Much easier solution.

I totally agree with your point that PDs should stop trying to game the system and use pure preference to determine their rank lists instead. I also agree that it's really dumb that how far you went down your rank list is a real measure of a successful match for programs. Like it's legitimately used to evaluate the performance of a program for some reason.

I wish there was another metric that could be used to evaluate the performance of a program...maybe ITE scores and board certification rates of graduates? Idk, lol

 

[Neopolymath]

Alternatively, they could just stop using "how far down a program goes on its rank list" as a status/prestige symbol and just let the Match algorithm work the way it's supposed to work. Then there'd be no need to mark people down or up based on the amount of interest they demonstrate. The entire Match ranking algorithm thing is ALREADY set up to take mutual interest into account when placing applicants...once PDs start trying to out-think the system they shank the whole damn thing.

Instead of tokens, just make it that PDs aren't allowed to share how far down the rank list they went, period. Much easier solution.


Sorry, but I can't really feel badly for a program getting 5 great incoming interns but their ego being a bit bruised in comparison to an applicant who would have been an excellent resident there and that the program would have loved to have getting shafted and changing their future career path (or potentially even not Matching) because PDs are trying to play a guessing game when there is literally a system ALREADY in place to take the guesswork out of it. Programs dipping to slot 30 to get the best shot at the best possible intern class they can have, while simultaneously NOT screwing applicants, sounds like a win-win to me.

We don't need a new solution. We need to actually USE the existing one.

This is going to require PDs to not be dinguses. Apparently, far more of them are ego-driven than we thought. I hope intern year is going well for you. I'm sure you are crushing it!

 

[Wordead]

I totally agree with your point that PDs should stop trying to game the system and use pure preference to determine their rank lists instead. I also agree that it's really dumb that how far you went down your rank list is a real measure of a successful match for programs. Like it's legitimately used to evaluate the performance of a program for some reason.

I wish there was another metric that could be used to evaluate the performance of a program...maybe ITE scores and board certification rates of graduates? Idk, lol

ITE scores are a measure of the individual residents it has nothing to do with the quality of a program. ITE is basically a crapshoot of questions that are only peripherally related to clinical practice, kinda like the steps. Programs care because how you do predicts if you'll pass your boards (which is important, but again, in my opinion is purely based on the work ethic of the person and has nothing to do with the program).

And I agree with you guys above. Program directors should just rank who they like in order. It's stupid to move people based on likelihood of them matching there. Though I do agree that there should be a way to weed out those who are applying just to apply - but I'm not sure this token system is going to work very well.

 

[NotaCop]

You guys see the drama with “Signal Tokens”? Basically a private company (charging applicants money) doing exactly this, tons of ethical concerns (one of the founders is a med student applying to residency this year, etc.).

Urology applicants got wind of it and weren’t happy with how the company was set up and how predatory it was looking, got vocal, and the AUA match people finally listened and issued a statement banning that company from including urology programs.

 

[mehc012]

ITE scores are a measure of the individual residents it has nothing to do with the quality of a program. ITE is basically a crapshoot of questions that are only peripherally related to clinical practice, kinda like the steps. Programs care because how you do predicts if you'll pass your boards (which is important, but again, in my opinion is purely based on the work ethic of the person and has nothing to do with the program).

And I agree with you guys above. Program directors should just rank who they like in order. It's stupid to move people based on likelihood of them matching there. Though I do agree that there should be a way to weed out those who are applying just to apply - but I'm not sure this token system is going to work very well.

It's called the interview. Seriously, that's a real time/money investment (tbd how COVID messes that up).
Other than that...why? It literally doesn't matter once they've taken an interview spot. Post-interview, nobody should need to signal interest.

 

[Wordead]

It's called the interview. Seriously, that's a real time/money investment (tbd how COVID messes that up).
Other than that...why? It literally doesn't matter once they've taken an interview spot. Post-interview, nobody should need to signal interest.

You think people don't go to interview at places theyre not really interested in?

I interviewed at several. As long as it doesnt conflict then people are going to go. Some go to nearly 20 which is crazy.

And now with virtual interviews I mean it's not even a money investment.

 

[longhaul3]

I went on 15 interviews. I would say I was very interested in 8 of those programs before I interviewed. The rest were just random invites I got that fit into my schedule around the ones I cared about. I would not have been thrilled if I had matched at any of them. There were programs that I was interested in that I chose not to interview at just because the dates available conflicted with the interviews I really cared about. (BTW, I think many ortho programs intentionally schedule their interviews on the same days to force applicants into signaling their true interest, which is savage but has the beneficial effect of reducing the number of interviews applicants have to go on).

I was interested in 3 programs by the time I finished interviewing. One of them was not in the original 8, and based on geography and my background I don't think they would have thought I was that interested. I sent the PD an email explaining why I was and ended up matching there.

Signaling interest goes a really long way in small specialties. It's nerve wracking and painful but I think it's really valuable post-interview. Programs want people who want to be there so they will work hard and not complain. When you have 2 interns a year, and one of them leaves because he hates it, the program is screwed. Not as much in medicine when you have 50 interns every year.

If I had signaled my interest before the interview trail, I would have wasted half my tokens on programs that I ended up not really caring much about. I understand the argument that it corrupts the calculus of the match, but there is no objective way to rank applicants anyway. I think it's totally legitimate for programs to rank people higher who are actually interested in the program—it's part of what makes an applicant desirable to a program.

 

[mehc012]

You think people don't go to interview at places theyre not really interested in?

I interviewed at several. As long as it doesnt conflict then people are going to go. Some go to nearly 20 which is crazy.

And now with virtual interviews I mean it's not even a money investment.

I think it doesn't matter. People are going to rank those places low, so they're less likely to end up at those programs. If you go on 20 interviews, and you weren't interested in some, they'll be ranked last. Who cares how interested they are when making the program's rank list? The only reason you would is to protect your yield.

Now, the choice of whether or not to INTERVIEW someone, there's a difference there. Interview slots are limited, and if you don't interview someone, you won't get them. But after the interview? Requiring applicants to signal interest is ridiculous and unnecessary, because the Match takes all of those things into account by having BOTH parties rank their preferences.

I would understand if there were more effort to figure out who to interview, but after that? No.

 

[efle]

I think it doesn't matter. People are going to rank those places low, so they're less likely to end up at those programs. If you go on 20 interviews, and you weren't interested in some, they'll be ranked last. Who cares how interested they are when making the program's rank list? The only reason you would is to protect your yield.

Now, the choice of whether or not to INTERVIEW someone, there's a difference there. Interview slots are limited, and if you don't interview someone, you won't get them. But after the interview? Requiring applicants to signal interest is ridiculous and unnecessary, because the Match takes all of those things into account by having BOTH parties rank their preferences.

I would understand if there were more effort to figure out who to interview, but after that? No.

The tokens system could have prevented so many one-that-got-away scenarios. There could finally be a way for high-caliber applicants to signal to mid-level programs that they truly have high interest there. I know some people write emails to PDs to express that but from the PD perspective who knows if they just send those to all programs?

I hope this spreads and sticks around, if it functions as intended