MelMcT2009

2+ Year Member
Jul 22, 2016
374
188
Status
Medical Student
Why do you guys think people do so much better on COMLEX compared to USMLE? I know several people who scored in the 50th percentile on USMLE, but 90-95th on COMLEX.

Yes, COMLEX is an easier test, but these tests are graded based on everyone's performance as a whole, so it shouldn't effect the score that drastically.
 

atomheart

2+ Year Member
May 5, 2016
224
332
Status
Resident [Any Field]
Where are your statistics coming from?
What's your sample size and what was the correlation coefficient?

Sent from my SM-J320V using Tapatalk
 
OP
M

MelMcT2009

2+ Year Member
Jul 22, 2016
374
188
Status
Medical Student
Literally every single person I've spoken with in my class who took both (around 10), myself (257/826), and pretty much all of the acounts I've read on SDN.

Definitely not a statistician by any means, but my own variation in scores is what got me thinking about it to begin with.
 

ortnakas

DO PGY-2
5+ Year Member
Jul 23, 2013
3,016
4,325
Status
Resident [Any Field]
I talked to a faculty member at my school about this-- percentile-wise I was definitely much better on the COMLEX the the USMLE. n=1, but this person's thought was that my school specifically tries to design COMLEX-style questions for our block exams, and designs a curriculum to prepare us for the COMLEX-- so we're obviously more ready for it than we are for the USMLE. They're evolving to prepare students for the USMLE as well, but perhaps not as quickly as they should be. It wouldn't surprise me if there's a similar situation at other DO schools.
 
OP
M

MelMcT2009

2+ Year Member
Jul 22, 2016
374
188
Status
Medical Student
I talked to a faculty member at my school about this-- percentile-wise I was definitely much better on the COMLEX the the USMLE. n=1, but this person's thought was that my school specifically tries to design COMLEX-style questions for our block exams, and designs a curriculum to prepare us for the COMLEX-- so we're obviously more ready for it than we are for the USMLE. They're evolving to prepare students for the USMLE as well, but perhaps not as quickly as they should be. It wouldn't surprise me if there's a similar situation at other DO schools.
That definitely makes sense that the schools would be teaching to the COMLEX. Thanks for the input!
 

Instatewaiter

But... there's a troponin
10+ Year Member
Apr 28, 2006
6,080
2,165
Washington
Status
Attending Physician
Why do you guys think people do so much better on COMLEX compared to USMLE? I know several people who scored in the 50th percentile on USMLE, but 90-95th on COMLEX.

Yes, COMLEX is an easier test, but these tests are graded based on everyone's performance as a whole, so it shouldn't effect the score that drastically.
This isn't rocket surgery. The cohorts are different. The COMLEX cohort is only DO students while the USMLE cohort is mainly US MD students and those DOs who opt to take it. The people who take the USMLE are smarter and are better test takers on average.
 
OP
M

MelMcT2009

2+ Year Member
Jul 22, 2016
374
188
Status
Medical Student
This isn't rocket surgery. The cohorts are different. The COMLEX cohort is only DO students while the USMLE cohort is mainly US MD students and those DOs who opt to take it. The people who take the USMLE are smarter and are better test takers on average.
I can agree with this to an extent, but I doubt it completely accounts for someone jumping from 50th to 95th percentile. Just too big of a jump
 

atomheart

2+ Year Member
May 5, 2016
224
332
Status
Resident [Any Field]
This isn't rocket surgery. The cohorts are different. The COMLEX cohort is only DO students while the USMLE cohort is mainly US MD students and those DOs who opt to take it. The people who take the USMLE are smarter and are better test takers on average.
Sadly, this is where the direction of the thread was going knowing SDN, which is why I expressed the need for hard data.

There is only data on those who have taken both, and for those people the correlation in scores is .8 which is strong. Needless to say, this tells us nothing about the strength of MD students taking comlex or the group of DOs that didn't take usmle. So even if there's a skew, it's unclear if it's the test, the test takers, or some combination. So you might be right, but there's no basis to make that conclusion yet.

http://www.ncbi.nlm.nih.gov/pubmed/20391276

Sent from my SM-J320V using Tapatalk
 
  • Like
Reactions: MelMcT2009
OP
M

MelMcT2009

2+ Year Member
Jul 22, 2016
374
188
Status
Medical Student
Sadly, this is where the direction of the thread was going knowing SDN, which is why I expressed the need for hard data.

There is only data on those who have taken both, and for those people the correlation in scores is .8 which is strong. Needless to say, this tells us nothing about the strength of MD students taking comlex or the group of DOs that didn't take usmle. So even if there's a skew, it's unclear if it's the test, the test takers, or some combination. So you might be right, but there's no basis to make that conclusion yet.

http://www.ncbi.nlm.nih.gov/pubmed/20391276

Sent from my SM-J320V using Tapatalk
Interesting. Even with this data the average USMLE was between 15th-18th percentile, and COMLEX was 70th.
 

Instatewaiter

But... there's a troponin
10+ Year Member
Apr 28, 2006
6,080
2,165
Washington
Status
Attending Physician
Sadly, this is where the direction of the thread was going knowing SDN, which is why I expressed the need for hard data.

There is only data on those who have taken both, and for those people the correlation in scores is .8 which is strong. Needless to say, this tells us nothing about the strength of MD students taking comlex or the group of DOs that didn't take usmle. So even if there's a skew, it's unclear if it's the test, the test takers, or some combination. So you might be right, but there's no basis to make that conclusion yet.

http://www.ncbi.nlm.nih.gov/pubmed/20391276

Sent from my SM-J320V using Tapatalk

You are making this much more complex than it needs to be to make yourself feel better. Name the defense mechanism...
 
OP
M

MelMcT2009

2+ Year Member
Jul 22, 2016
374
188
Status
Medical Student
You are making this much more complex than it needs to be to make yourself feel better. Name the defense mechanism...
I don't think it's as simple as you are trying to make it either. The correlation between USMLE and COMLEX is all over the place.

I'm not trying to (nor do I need to) make myself feel better about anything. Just a question I've had and I'm sure many others have as well. No need for you to respond if you have nothing to contribute.
 

atomheart

2+ Year Member
May 5, 2016
224
332
Status
Resident [Any Field]
You are making this much more complex than it needs to be to make yourself feel better. Name the defense mechanism...
I'm not disputing the quality of DO students taking only comlex, just as I'm not disputing the quality of Caribbean students taking only usmle.

The quality of the test matters. It's in everyone's best interest to be able compare apples to apples.

Sent from my SM-J320V using Tapatalk
 
Last edited:

Instatewaiter

But... there's a troponin
10+ Year Member
Apr 28, 2006
6,080
2,165
Washington
Status
Attending Physician
I don't think it's as simple as you are trying to make it either. The correlation between USMLE and COMLEX is all over the place.

I'm not trying to (nor do I need to) make myself feel better about anything. Just a question I've had and I'm sure many others have as well. No need for you to respond if you have nothing to contribute.
The correlation isn't "all over the place". In the link above the correlation coefficient is 0.85. There are few things that have that strong of a pearson correlation. Above .8 is considered very strong correlation so basically it is the opposite of "all over the place".
 

Instatewaiter

But... there's a troponin
10+ Year Member
Apr 28, 2006
6,080
2,165
Washington
Status
Attending Physician
I'm not disputing the quality of DO students taking only comlex, just as I'm not disputing the quality of Caribbean students taking only usmle.

The quality of the test matters. It's in everyone's best interest to be able compare apples to apples.

Sent from my SM-J320V using Tapatalk
So the reported mean to which the USMLE percentiles come from are only from US/Canadian seniors. The USMLE mean does not include FMGs/IMGs (and probably not DOs).

http://www.usmle.org/pdfs/transcripts/USMLE_Step_Examination_Score_Interpretation_Guidelines.pdf
 

runthiscity

5+ Year Member
Jul 8, 2013
84
72
Status
Medical Student
http://www.jgme.org/doi/pdf/10.4300/JGME-D-13-00302.1

Here's an article publsihed comparing students who took both USMLE and COMLEX (n = 1016). Although the goal of the article was not only to compare COMLEX vs USMLE scores, but also determine if various "score conversion" formulas available were actually valid. There's a nice figure in there showing USMLE vs. COMLEX scores that's worth a peak. They found that you really can't use any score converter, and- as @atomheart said- it is best to compare apples to apples. Different number of questions, different content (I think USMLE is better at testing on more discrete subjects where COMLEX seems to hammer the same subjects more than others.)

I'm another n=1, but my USMLE and COMLEX percentile scores are way off from one another, with my COMLEX being far higher (*shocking* :eek:) However, from looking at FREIDA, it seems most residency programs basically account for this themselves. For residencies listing the Step 1 average as 221-240, they frequently have the COMLEX level 1 average as 551-650. FREIDA likely isn't the best source of information on the issue, since it's up to the PDs to select the ranges, and they're just pre-set ranges at that. Also, some of those programs take USMLE only yet list COMLEX anyway; these students likely self-selected as having a better shot of doing well on the USMLE to begin with.

Finally, you can't really compare DO students taking both USMLE and COMLEX to MD students only taking USMLE for just that reason. MD students don't take the COMLEX. For all we know, they could score lower on average on the COMLEX than their DO counterparts. This is unlikely since they're statistically more likely to be better test takers, but nonetheless, it cannot be ruled out. They could score such that there is no significant difference between MDs and DOs, or they could outscore DOs as they tend to do on the USMLE. However, until some MD takes the COMLEX (controlling for the OMT shenanigans, which is a whole different can of worms) there is really no way to accurately compare between scores.

This leads into the whole other argument- should there be a single exam for both MDs and DOs, with DOs having an extra OMT bonus section of sorts. Arguments can be made easily for one side or the other- we're moving towards single accreditation, we should have one licensing exam vs. this would probably stratify students the same way as the MCAT did, MD and DO are different enough curriculums that they should not take the same test.

All that aside, the whole MD vs DO/ USMLE vs COMLEX debate really grinds my gears at times. It's important to note that I likely feel this way because my 230 is not always going to be viewed the same as an MD's 230, so I got the short end of the stick here and I'm bitter. Boo hoo hoo. Ideally, everyone will remember that who you will be as a doctor comes down to way more than your USMLE/COMLEX score. Let's be real, some of us were just not cut out to be neonatal dermatologists when we came out of the womb and our APGAR score was only an 8. You could be dumb as a sack of rocks and have patients who think you're awesome because you take the time to listen and hear them out. You could also be a total USMLE ~*RoCkStAr~* and still have patients who love you for the same reasons. And patients can dislike you as a doctor whether you're subjectively smart or dumb as well. I'd be willing to bet that 99% of the time, people don't chose where to live based on what hospital they're next to ("ohhh this is the best [insert specialty here] residency in the country, I will only live within driving distance of this hospital!" ... I don't think so.) Patients likely don't give two ****s about what your step score was or where you went to school, as long as you make them feel like a human and make them "better."
"But runthiscity! It matters for specialties!!" Yes, yes it does. And you better know your **** if you plan on being a good doctor no matter what specialty you do. But at the end of the day, the comparison of scores all just a pissing contest to make ourselves feel superior to our colleagues, and those in other professions- because med school is just *the hardest* and no one else could *ever* take board exams like we do. "But runthiscity! Your scores matter for where you get interview offers! And thus, where you end up living semi-/totally- permanently!" Yes, yes they do. And how else are programs going to sift through thousands of applications to figure out who to interview? So, we must play the game. But I would personally prefer someone who got a 220/530 and is compassionate than someone who got a 276/920 and has an ego the size of Jupiter. But that could just be me.
/rant