pkwraith

7+ Year Member
Apr 15, 2011
768
75
Status
Resident [Any Field]
What is the purpose of this project?
To help provide some guideline to future (and current) EM Residency applicants early in the Interview Season. After application submission, applicants are often uncertain about how many interviews they should be receiving. The uncertainty is probably made worse when they see a small minority receiving large amounts of interviews. The ultimate goal is to allow applicants to see where similar applicants place at similar time and give context to their current position and trajectory. Similar to Charting the Outcome, except for Interviews.

Why should you help?
If at any time this interview season, you wish you knew where you stood with your peers, this will help others like you in the future.
Or if you are just tired of panic posts from other neurotic applicants and could just post an link/graph.
Or if you just like looking at data, statistics, and making sense of it.

How is this project done?
We will be using a public google spreadsheet:

https://docs.google.com/spreadsheets/d/1kK8NYX6rDOjgvWMOQO9xBVKTCs_ackeYN5BuZ-uBDp8/edit?usp=sharing

A few notes. This is intended to be anonymous so people are more willing to contribute. The identifier is for YOUR purpose, to keep track of which row contains your data, so put something that you can associate with yourself (like anonymous + unique ID). There is no school identifying data -- simply put in the dates of interviews as you received them. This is open to anyone, not just SDN posters, if you want to help out and put in your non-identifiable data.


What sort of data will be collected?

Right now, the stratifying criteria will be Step 1 Score and US Senior vs Independent. There are obviously other components of application, but in the interest of preventing scope creep, they can be certainly be considered in the future.


How will the data be presented?
There will be a few different presentation. But, an example mockup is the following:

Interview fraction is (number of interviews received ) / (number of application submitted). Someone received 30 interviews of 30 applications is not equivilant to someone receiving 30 interviews of 100. This allows me to compare and aggregate the interviews received.


How do I know you're not making everything up?
You don't. It's the internet, and the data is based on a public spreadsheet. In the interest of transparency, I will provide the spreadsheet of my analyses on request for people interested in doing data crunching, but obviously neither I nor anyone can control what people put in the spreadsheet, and this is all on honor system. Having additional constraints will just result in less participation.

Interpretation of Data:

<Various graphs will be here>

And to get the following out of the way:
Being above your interview curve does not mean you will match. Being below your interview curve does not mean you will not match. However if you are very far below your curve, it is a definite red flag to review your applications with someone. In the end, nothing beats an experienced adviser who can look over your list and select your schools.
 
Last edited:
  • Like
Reactions: emememem

link2swim06

10+ Year Member
Dec 14, 2007
3,278
792
Status
Attending Physician
You should average the step 1 and 2 scores. EM actually cares about step 2; probably more so than step 1.
 
OP
pkwraith

pkwraith

7+ Year Member
Apr 15, 2011
768
75
Status
Resident [Any Field]
You should average the step 1 and 2 scores. EM actually cares about step 2; probably more so than step 1.
For many to most people, Step 2 Scores are not back by the start of submission, and many programs send out interviews before they see Step 2. I also suspect there is a correlation between Step 1 and Step 2 as well.

Not to minimize the importance of Step 2 (and other stuff like LORs), but they're harder to factor into interviews than Match, when programs can seen Step 2 when they are ranking.

love how everyone trends towards the bottom for interviews unless you have a 250+ lol.
That's just a rough mockup of the format of results. It is entirely possible that Step 1 scores don't stratify as much as I exaggerated!
 
Last edited:

hoot504

10+ Year Member
Jan 17, 2009
457
179
Status
Attending Physician
What exactly is 'interview fraction' representing?
 
OP
pkwraith

pkwraith

7+ Year Member
Apr 15, 2011
768
75
Status
Resident [Any Field]
What exactly is 'interview fraction' representing?
Fraction is (# of interviews received)/ (total # applied)

absolute # of interviews is not a great measure to me, since someone who applies to 80 programs should have more interviews than a person that applied to 15. Using fraction is a way of controlling for people's different numbers of total applications.

So, ideally, someone would say "Okay, I applied to 30 programs. Most people similar to me received about 30% of their interviews at this time, so an estimate would be around 10 interviews +/- 3."
 
OP
pkwraith

pkwraith

7+ Year Member
Apr 15, 2011
768
75
Status
Resident [Any Field]
Hey guys, appreciate everyone who took the time to add some data in. I know Google Spreadsheet has been bugging out in the last few days. Hopefully they get it fixed soon.


So, just taking an early look at the data. I'm using the 231-340 group right now because it's our biggest group.

Early data analysis is very encouraging:


As you can see, 3 out of 4 people in the subset has very similar interview curves. More clustered than I had hoped for, giving all the other factors. Very, very promising.

Here is the Averaged out curve (including the outliers):



Obviously, given the low sample size, the actually number results at this point in time is not optimal. But, the pattern and shape is there, and I don't expect that to deviate much from the final results.

If there is enough sample size, I would actually like to be able to cover quartiles and standard deviations.

If you're not one of the 231-240 group and is curious to see what your curve is like, I highly recommend you help by adding your own data. I think this can turn out to be very cool, but it's all your hands. Thanks.
 
Last edited:
OP
pkwraith

pkwraith

7+ Year Member
Apr 15, 2011
768
75
Status
Resident [Any Field]
Lots of thanks for everyone who have put in their data, and the people that are keeping theirs updated.

I haven't forgotten about this. I've been really busy with my wards. Here is a latest update (as of Oct 31):



A few notes. Small n right now, so data are subject to huge swings from outliers. I've tried to make as much adjustments as I can, but the most important thing is having more datapoints.

Once more data is in, my next steps are to work on Quartile data (25%, 75%, median), as I think that would be more helpful guidance, aka, if you are below the 25 percentile mark, you may want to consider adding more interviews.

I'll put out some Independent applicant analysis out soon as well.
 

discostu42

5+ Year Member
Dec 11, 2013
33
25
i applaud your efforts, and think this a great project for future applicants. As a DO applicant for ACGME progs, I think future kids will wanna know the breakdown of the independent category, whether IMG/FMG/DO so that there can be more transparency
 

AvocadoLover

10+ Year Member
7+ Year Member
Apr 15, 2007
152
12
California
Status
Resident [Any Field]
i applaud your efforts, and think this a great project for future applicants. As a DO applicant for ACGME progs, I think future kids will wanna know the breakdown of the independent category, whether IMG/FMG/DO so that there can be more transparency
I concur, this would be very helpful! Thank you!
 
OP
pkwraith

pkwraith

7+ Year Member
Apr 15, 2011
768
75
Status
Resident [Any Field]
I would not have any problems with people putting down DO/IMG/Other and tracking the ACGME interviews. Although, we would need more DO guys to put down data, as I don't think there's much to work with currently.
 
OP
pkwraith

pkwraith

7+ Year Member
Apr 15, 2011
768
75
Status
Resident [Any Field]
I said I'd something with IMG data, but frankly, we don't have enough of it.

I threw together something from what was there.


Aggregate interview curves US Senior vs Independent



US Seniors (by Step 1) vs Independent

Nothing that we don't really know. Independent Interview rates are significantly lower in aggregate vs US Senior, and lower than the lowest Step 1 level of US Senior. And therefore they require significantly more applications.

However, from what I'm seeing, there is significant stratification in the Independent group, and less precision, so there is probably more variables at play there than with your standard US Senior (which really is evident).

If you're an Independent and want more useful data, than by all means, contribute. If you've already contributed, I think it makes sense to go ahead and put in DO/IMG in the US Senior/Independent field.