Question for aPD

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

Abram Hoffer

Full Member
10+ Year Member
15+ Year Member
Joined
Aug 22, 2008
Messages
195
Reaction score
0
"In my experience, USMLE performance clearly predicts residency performance -- those with low scores do not do well." -aPD

This is a comment aPD made on another post. I did not feel that my post belonged in the OP because it is an unrelated issue.

Would you (aPD) elaborate on your theory please? I would like to understand what makes an exception to this general rule.

For example, I worry a bit now about being a successful resident because I fall into your low score, poor resident theory, though I would like to believe that I will be the exception to the rule.

What are those performance measures that show those who do not do well?

What should I do to compensate for my weaknesses?

Thank you in advance.

Members don't see this ad.
 
Last edited:
"In my experience, USMLE performance clearly predicts residency performance -- those with low scores do not do well." -aPD

This is a comment aPD made on another post. I did not feel that my post belonged in the OP because it is an unrelated issue.

Would you (aPD) elaborate on your theory please? I would like to understand what makes an exception to this general rule.

For example, I worry a bit now about being a successful resident because I fall into your low score, poor resident theory, though I would like to believe that I will be the exception to the rule. Thank you in advance.

I'm not a program director but I can say that I have not seen individuals with low USMLE scores do worse in residency.

I have spoken to program directors about the issue. Most that I spoke with did not agree that low scores meant poor residency performance.

So I talked to a couple program behavioral directors (they are the ones that measure and keep data on testing).

They sited a couple studies that demonstrated no links between test scores and residency performance. (sorry I don't have those studies we were just having a casual conversation and did not take the time to go back and research the studies).

But your post brings up something more important abram Hoffer. It shows that you are setting yourself up for failure based on one persons belief or assessment.

I would suggest instead of focusing on your poor performance you focus on going back and learning the topics you did poorly on in the USMLE.

Also pick up some books that are focused on the first year of residency and read them.

Residents guide to ambulatory medicine
Washington manual (actually this book can come in handy anytime)
Read, Read, Read your specialty journals. If you are going into Family medicine start reading the AAFP journal and do the cme questions. You can also join the AAFP and do all the old questions and read all the previous issues.

15 to 20 minutes per day of reading. One to two articles per week. That will increase your knowledge base significantly.

Also take some classes on improving your test taking skills.
 
I'm not a program director but I can say that I have not seen individuals with low USMLE scores do worse in residency.

I have spoken to program directors about the issue. Most that I spoke with did not agree that low scores meant poor residency performance.

So I talked to a couple program behavioral directors (they are the ones that measure and keep data on testing).

They sited a couple studies that demonstrated no links between test scores and residency performance. (sorry I don't have those studies we were just having a casual conversation and did not take the time to go back and research the studies).

But your post brings up something more important abram Hoffer. It shows that you are setting yourself up for failure based on one persons belief or assessment.

I would suggest instead of focusing on your poor performance you focus on going back and learning the topics you did poorly on in the USMLE.

Also pick up some books that are focused on the first year of residency and read them.

Residents guide to ambulatory medicine
Washington manual (actually this book can come in handy anytime)
Read, Read, Read your specialty journals. If you are going into Family medicine start reading the AAFP journal and do the cme questions. You can also join the AAFP and do all the old questions and read all the previous issues.

15 to 20 minutes per day of reading. One to two articles per week. That will increase your knowledge base significantly.

Also take some classes on improving your test taking skills.
Thank you for posting your comments. While I posed a specific question to a specific person, I wanted others would chime in with their thoughts on the matter.

I am not worried about a negative self-fulfilling prophecy. I do feel that, if this is a general rule, I am an exception to it. I am quite confident in my abilities, but I would like to understand why people who hold this theory do so and what they suggest makes a strong resident.

Part of what I know is that I have worked with residents who had solid scores, and they weren't the most impressive individuals. But the things that some value are not necessarily the same things others value.
 
OK, I'll bite.
I'm someone who had pretty good USMLE scores, but not through-the-roof. My Step 2 and step 3 scores were actually pretty high, with Step 1 being average. I took the average Step 1 as a warning sign that I needed to study harder for Step 2 and Step 3, but also that perhaps I had some knowledge gaps and also needed to brush up on my test taking skills.

I have a friend who has very very high USMLE scores and did VERY well on our medicine in-service exams and the ABIM test (test you take to get board certified in IM). I do think she was a very good resident, particularly knowledge-wise, and everyone agreed she came across as very smart.

There are reasons besides lack of knowledge and/or ability to problem solve that might explain why a person had a low test score. I know that for some of my friends for whom English is not the first language, they never score that high on these types of tests, likely because it takes them longer to read and comprehend the questions. Also, anyone can have a bad day - the test just reflects your performance on that day. Some people just aren't good at taking multiple choice standardized tests, either, and perhaps the test results doesn't totally reflect what they know because of that.

There is no doubt in my mind that USMLE scores have some correlation with level of knowledge and with problem solving in the context of medical treatment, and that more knowledge and better problem solving ability makes it possible to make better clinical decisions, all else being equal. Notice that I said ALL ELSE BEING EQUAL. Of course we all know that all else is NEVER totally equal. Being able to relate to your patients and their families, and actually liking the patients and knowing how to make them like you, as well as being able to explain things to them ( = "bedside manner") is important in all the patient-contact specialties, particularly in things like primary care and psych. Persistence, dedication, having a commitment to medical ethics, having a commitment to lifelong learning - these things are important too. Critical, actually.
 
Reviewing my comment, I can see how it can be misinterpreted. I believe you have made a logical error in your assumptions (which is a good thing!). So let me try again.

As previous posters on this thread have noted, studies of this have shown no correlation between USMLE scores and resident performance.

In my single program experience, here's what I have noticed:

1. First, I run a tight ship and the number of residents who end up struggling is quite small. So, it's hard to extrapolate.

2. Residents who struggle because of professionalism or communication skills issues often have fine USMLE scores.

3. Residents who struggle with Patient Care issues often have low USMLE scores, usually on Step 2. Step 1 scores do not seem to matter, other than they are related to step 2 scores (i.e. people who score lower on Step 1 more often (but not always) score lower on Step 2 also). BUT...

4. Most residents with low-ish USMLE scores do just fine.

So, my original statement is true -- most of my residents who have struggled have low USMLE scores. However the opposite is NOT true: That residents with low USMLE scores all struggle. Many do just fine.
 
Reviewing my comment, I can see how it can be misinterpreted. I believe you have made a logical error in your assumptions (which is a good thing!). So let me try again.

As previous posters on this thread have noted, studies of this have shown no correlation between USMLE scores and resident performance.

In my single program experience, here's what I have noticed:

1. First, I run a tight ship and the number of residents who end up struggling is quite small. So, it's hard to extrapolate.

2. Residents who struggle because of professionalism or communication skills issues often have fine USMLE scores.

3. Residents who struggle with Patient Care issues often have low USMLE scores, usually on Step 2. Step 1 scores do not seem to matter, other than they are related to step 2 scores (i.e. people who score lower on Step 1 more often (but not always) score lower on Step 2 also). BUT...

4. Most residents with low-ish USMLE scores do just fine.

So, my original statement is true -- most of my residents who have struggled have low USMLE scores. However the opposite is NOT true: That residents with low USMLE scores all struggle. Many do just fine.

I would agree with that statement. It is about the same as when I spoke with the above mentioned individuals.

You did say it better.
 
Reviewing my comment, I can see how it can be misinterpreted. I believe you have made a logical error in your assumptions (which is a good thing!). So let me try again.

As previous posters on this thread have noted, studies of this have shown no correlation between USMLE scores and resident performance.

In my single program experience, here's what I have noticed:

1. First, I run a tight ship and the number of residents who end up struggling is quite small. So, it's hard to extrapolate.

2. Residents who struggle because of professionalism or communication skills issues often have fine USMLE scores.

3. Residents who struggle with Patient Care issues often have low USMLE scores, usually on Step 2. Step 1 scores do not seem to matter, other than they are related to step 2 scores (i.e. people who score lower on Step 1 more often (but not always) score lower on Step 2 also). BUT...

4. Most residents with low-ish USMLE scores do just fine.

So, my original statement is true -- most of my residents who have struggled have low USMLE scores. However the opposite is NOT true: That residents with low USMLE scores all struggle. Many do just fine.
If an applicant comes into your office for a residency interview and is questioned about their lower USMLE scores, what is it that you are seeking from that question? I realize this may be candidate-specific, but... Is it that they learned from it? Is it that they figured out a better way to overcome obstacles? Is it that they weren't knocked over by a failure (or lack of success)? Something else?
 
"In my experience, USMLE performance clearly predicts residency performance -- those with low scores do not do well." -aPD

There was a study done in 2007 that showed a significant correlation between Step 1 performance and passage of the Pediatric board examination. The authors speculated that by knowing that poor Step 1 performance correlates with failure of the Pediatric board then these residents could have intervention early in residency to help them study better.

In the end, clinical evaluation anywhere is subjective, and sure I have seen students who could smooze their way to a passing grade even while having failed important examinations. The same thing happens in residency where soemtimes residents who act very competent are not. There is a reason for having inservice exams as attendings in general have pretty subjective evaluations.

In Ob/Gyn those with a board score above 200 were seven times as likely to obtain board certification than those with a score below 200.

So, I think that USMLE is good in that it is an objective measure of clinical knwoledge and clinical management.

I think that some residents can eventually and easily outperform their Step 1/2 score because time is on your side. If you read everyday and read about your patients then you will catch up with your fellow residents who by and large studied more efficiently than you in medical school. This is an uphill battle, but such motivated students are doubtlessly spotted by residency program directors who want someone who will study in residency and plough through the material.

Most excellent internal medicine programs want applicants with relatively high Step 1 scores, i.e. above 230, and nowadays having a high Step 2 CK helps as well.

Doing a study in such a population of residents at MGH in terms of correlation between board score and performance in residency and comparing the results to other institutions is flawed. A mediocre or lukewarm good resident at MGH likely would be a superstart at Podunk Internal Medicine residency.

On the whole residents at big academic centers with high Step 1's really know what they are doing, but some even there some stand out. So if you have someone at Podunk Internal Medicine program with an average board score who is rated as "excellent" by faculty versus someone who has 250 at MGH who is just "average" then it would look like there is no correlation between board scores.

I have seen interns who didn't make it in residency and they had knowledge deficits and poor clinical solving skills, i.e. poor Step 2 CK scores as well as no motivation. There is a study of students who failed Step 1, I will try to find the info. In the end, a PD who asks about a student who failed Step 1 is really trying to see if the student can rise above it as a resident. Step 1 is not rocket science or neurosurgery, if you fail this exam you probably weren't motivated to study medicine, or had a pretty big extenuating circumstance.

In the end I think there are cohorts:

1. Failed Step 1 (or 2). This student needs to re-motivate themselves to learn medicine, and will be judged by Step 2 CK score and clerkship grades and can redeem themselves, but motivation judged in residency interview.

2. Step 1 (or 2) - 187-200. Just passed, obviously below average. Need to also look at clerkship grades and letters to gauge level of committement and to see if the residency program can help such a student bloom into an excellent doctor. I think interviewers try to see if the student understand what went wrong and how to fix it. More high-pressued residencies like surgery, top-notch IM might not benefit such a student as much as others, but motivation for field still a factor. Some residencies will filter out such applicants as the program moves at a fast pace and such a resident may need extra time to figure out what they don't know.

3. Step 1 (or 2) 200-230 This student had some level of passion for medicine, has an adequate knowledge base and could become excellent in residency if he/she learns their field well and is diligent when taking care of patients. Residency program directors like to see how motivated such a person is for say pediatrics. If applicant says that it took them a while to get down level of studying needed for med school, but love their peds core and working with kids then they would be expected to do well.

4. Step 1 (or 2) 230-250 Something really clicked well for this student so he/she could really learn some medicine. Major question is "fit" of the resident with the program, i.e. can they reproduce this result in the environment of your residency program.

5. Step 1 (or 2) > 250. These students by a large number self-select to competitive surgical residencies, dermatology, and top-notch medicine programs. Generally will grows roots and bloom wherever they are planted, or so is assumed. Also want to see if has similar personality to residents.

I think if a high Step 1/2 CK score reflects a lifelong commitment to learn medicine then it is significant especially with medicine becoming more complicated by the day and requiring more reading to stay competent.

If most of a program director's residents are in Group 2 or 3, then there might not be much correlation between residency performance and usmle, although a lot of studies show a correlation between one or the other of the steps and inservice training examinations.
 
Last edited:
interesting theory, save:

1) grouping scores from 200 (1 SD < mean) to 230 (1 SD > mean) is very, very silly, as that captures 67% of all test-takers;

2) most performance-based as opposed to cognitive specialties say Step 2 performance is most highly correlated with both clinical success and performance on in-service and board exams;

3) a large percentage of people who do very well on Step 1 prefer the non-clinical specialties (I know at Columbia, more than half of the people in the 270s end up in radiology)
 
interesting theory, save:

1) grouping scores from 200 (1 SD < mean) to 230 (1 SD > mean) is very, very silly, as that captures 67% of all test-takers;


2) most performance-based as opposed to cognitive specialties say Step 2 performance is most highly correlated with both clinical success and performance on in-service and board exams;

3) a large percentage of people who do very well on Step 1 prefer the non-clinical specialties (I know at Columbia, more than half of the people in the 270s end up in radiology)

I am just guessing based on observations in terms of the general groups of students. I don't think it is silly to differentiate people into a 200 - 230 group as this is the middle of the bell curve, 67% sounds good to me. And maybe it sounds good to some IM program directors of top notch programs too who do use 230 as a cutoff. Step 1 is more correlated with pediatric board passage than Step 2, but again someone who does well on Step 1 has a good chance of doing well on Step 2. I think Step 2 is important as well though.

While radiology may be a "non-clinical" field, radiologists interact with internists, surgeons, pediatricians all the time and a great radiologist can mean the difference in making the right treatment decision in a lot of cases. Not that I am a radiologis! But, some radiologist do IR which is pretty clinically involved, i.e. like surgery, so I wouldn't say that scoring high on Step 1 and doing radiology means Step 1 doesn't work or whatever your point was as many radiologists are excellent physicians that add to patient care in real ways.

I don't think there is a big difference between a 220 and a 205 . . . perhaps saying a 200 is the same as a 230 is silly, but you have to draw the line somewhere, and above 230 opens more doors for you as a residency applicant for sure.

Most specialties are "cognitive" specialties such as internal medicine, pediatrics, family medicine, need I go on? So, based on your statement then Step 1 would be imore important for these than Step 2?

Realize that "performance based" specialties like orthopedic surgery give a lot of weight to a high Step 1 score, and Step 2 is less of a factor for most residency applications as many applicants don't have these scores, although this may change as I think Step 2 is more reflective of success specialties like internal medicine.

But it would be a good idea if someone compiled a list of step-cut offs that some residency programs use, for example:

1. Ob/Gyn, some programs I have heard have a Step 1 cutoff of 200 or even 210
2. Some internal medicine programs have a Step 1 cutoff of 230, or even use 235+
3. Some opthalmology programs had used 220+ cutoff before getting competitive I guess.
4. Even some family practice residency programs may have a cutoff of 195.
5. In dermatology some applicants believe that you need a 240+ with publications.
 
We now know that Darth has excellent USMLE scores :)

Some good points made by numerous folks on this thread.
For all you applicants - use caution in swallowing anybody's "cutoffs" for USMLE for entering certain residencies. I mean, it's definitely true you're not getting in derm with a 205 on USMLE step 1. However, this 230 that is bandied about as being a cutoff for certain IM programs, etc. I don't think is necessarily true.

I only got 224 (or was it 225?) on Step 1 and was not in the top 1/2 of my medical school class and I got interviews at competitive university IM programs (i.e. Washington U, Duke, Emory, Vanderbilt, Case Western, etc.) and attended one. I didn't even honor my medicine rotation during 3rd year, although I did honor my medicine subI. I think there are times when people deviate from these supposed cutoffs, and I also don't think that the cutoffs for things like peds, psych, IM, even at the "top" programs are as high or as fixed as what some people on SDN are contending on a regular basis. I think that other things in an application, such as the pretty extensive leadership and extracurricular activities I had, a research publication like I had, a Dean's letter with lots of positive comments, etc. are also factored in and not as many programs in the less competitive specialties actually "filter" using the 230 USMLE as some people seem to think.

I also don't think a 200 on the USMLE is very similar to a 220 or 230...
I actually got 240's on Step 2 and 3 and I do think I had a better knowledge base and preparation than I had on Step 1, but not necessarily by much. I think rather than dividing USMLE scorers into groups (i.e. <200, 200-230, 230-250, >250, etc.) one should realize this is a continuous distribution of scores. It's a bell curve. So while there is a difference between 200 and 230, there's not much of a difference between a 229 and a 231. I think if I were a PD and wanted to use a "filter" I'd probably set it around the average USMLE score, or even a bit lower, unless this was for one of the most competitive of competitive specialties (in which case they just need multiple ways to filter out some of the many applications). My thinking would be I'd rather have the applications to look at so that I could "filter" using multiple things, not just a USMLE score.
 
But it would be a good idea if someone compiled a list of step-cut offs that some residency programs use, for example:

1. Ob/Gyn, some programs I have heard have a Step 1 cutoff of 200 or even 210
2. Some internal medicine programs have a Step 1 cutoff of 230, or even use 235+
3. Some opthalmology programs had used 220+ cutoff before getting competitive I guess.
4. Even some family practice residency programs may have a cutoff of 195.
5. In dermatology some applicants believe that you need a 240+ with publications.

As you know, these data will be almost imposible to obtain.
 
(I know at Columbia, more than half of the people in the 270s end up in radiology)

How many 270+ does columbia have? Lurkering on SDN since I was a premed I've only known of 2 people with verifiable 270+ and one is going into rads.
 
How many 270+ does columbia have? Lurkering on SDN since I was a premed I've only known of 2 people with verifiable 270+ and one is going into rads.


I know two just from my med school. One from my class, one matched directly into Rad-Onc (my class). The other is a GS categorical intern. There are rumors there were two-three more so a total of about 4-5 in three years from a school that only graduates 60 per year.

I would suspect that if our little school had that many (again I only know two for a fact but the others were plausable) then I would guess Columbia has a good number of them.
 
As you know, these data will be almost imposible to obtain.

Which is absolute crap. Program Directors should state on the application guidelines the cutoffs they use (yeah yeah they'll tell use they don't use a cutoff because they're always that potential applicant that score 185 on both Step 1 and 2 that cured AIDS and Cancer while kayaking in a third world country). While applying, it's easy to tell which programs have a cutoff when you get a rejection letter a few days after submitting the application early. Guess the money I could have saved by not applying to your program won't be going toward my mounting student loan debt.
 
Which is absolute crap. Program Directors should state on the application guidelines the cutoffs they use (yeah yeah they'll tell use they don't use a cutoff because they're always that potential applicant that score 185 on both Step 1 and 2 that cured AIDS and Cancer while kayaking in a third world country). While applying, it's easy to tell which programs have a cutoff when you get a rejection letter a few days after submitting the application early. Guess the money I could have saved by not applying to your program won't be going toward my mounting student loan debt.
There's not a process in the world that cannot be improved--ERAS included. While there are many positives about the ERAS, there are many things that need to be improved. How difficult it would be to gather this info, put together a committee to implement these changes, etc, lies in the hands of those who think that there need not be any changes. There are many talented people on this forum alone who have made suggestions that would make ERAS so much better. The hardest part is to be invested beyond the current year, especially after realizing how much time and money could have been saved.
 
2. Residents who struggle because of professionalism or communication skills issues often have fine USMLE scores.

What are some of these professionalism issues? Can you give some examples? I genuinely have no idea, since definitions vary so much.
 
What are some of these professionalism issues? Can you give some examples? I genuinely have no idea, since definitions vary so much.

1. Lying to me about important things. Like where you were when you were supposed to be at work.

2. Getting into a fistfight with hospital security.
 
1. Lying to me about important things. Like where you were when you were supposed to be at work.

2. Getting into a fistfight with hospital security.

Both of those are unprofessional but I have to ask. Are you talking about the ex police six foot + hospital security or the kind that look like they failed out of mall cop school? There's a difference.

David Carpenter, PA-C
 
1. Lying to me about important things. Like where you were when you were supposed to be at work.

Oh please, how would you find out? :) Do you find that being a program director is a lot like being a parent? :laugh:
 
Oh please, how would you find out? :) Do you find that being a program director is a lot like being a parent? :laugh:

Running into your Assistant PD at the mall when you are at home "sick" is usually one way.
 
This seems acceptable under man-rules. Who won?

Its kind of like getting in a fight with the police. You may "win" the first round but the next 10 or so will probably end it for you.
 
Oh please, how would you find out? :) Do you find that being a program director is a lot like being a parent? :laugh:

When you get initiated into the secret society of program directors, you get a special crystal ball which allows you to see what all of your trainees are doing at any time. Comes in handy. I've wired it to the USB port of my computer, so I stream the video feeds and review them regularly.

Plus, we implant GPS chips into your skulls on your first call night. Remember that headache you woke up with?
 
Comes in handy. I've wired it to the USB port of my computer, so I stream the video feeds and review them regularly.

Plus, we implant GPS chips into your skulls on your first call night. Remember that headache you woke up with?

One hospital I know toyed with putting GPS locators in the pagers they give residents, by going to an intranet webpage you could get a map of the hospital with icons for the various residents and attendings. I am sure you could program the system to calculate percentage time spent in the caf or break room, given enough money. Would save a ton of time for surgical residents/attendings so you could track people down easier. The technology is there to do this stuff.
 
When you get initiated into the secret society of program directors, you get a special crystal ball which allows you to see what all of your trainees are doing at any time. Comes in handy. I've wired it to the USB port of my computer, so I stream the video feeds and review them regularly.

Plus, we implant GPS chips into your skulls on your first call night. Remember that headache you woke up with?

In psych we just tap our magic map with our wand and state "I solemnly swear that this patient is suicidal with a plan".
 
How many 270+ does columbia have? Lurkering on SDN since I was a premed I've only known of 2 people with verifiable 270+ and one is going into rads.

Not that many. 2-3 per year. Our dean of students releases this absolutely wonderful, reassuring document that give anonymized data on the boards scores, honors in core rotations, research year, etc data for each successfully matched candidate per specialty. So this is cold, hard data, not a murkily cobbled together collection of anecdotes.

Interestingly most of the other super-high scorers end up in medicine or neurosurgery-- though, as I said, more than half end up in radiology.
 
Top