MCAT is correlated to Step 1 scores.

[libertyyne]

---

Members don't see this ad.

That's really the kicker, I think. With the increased opportunities for networking and increased access to resources for academic support at top programs it makes sense that it would be easier for students at such institutions to achieve higher USMLE scores (and they would have had higher MCAT scores to get into said institutions) and thus more competitive/lucrative residency placements. But that's really just a population trend, and doesn't necessarily have to apply to individuals. If a student was able to succeed through standardized testing and make the proper connections while going to a mid-tier school, they too can achieve high scores and placement in competitive residencies, albeit the road was more difficult for them. The inverse could also be true, with high-achieving undergrads going to top programs only to fizzle out and not putting in the necessary work or taking advantage of their resources, and thus ending up with a low-paying position later in life.

Scores could be correlated with income, sure. But, to propose a correlation between academic institution and income would not work out very well, particularly with the already moderate correlation between various tests.

This exists for UG already. Mostly due to the fact that top UG institutions also tend to feed into professional programs. I am still waiting for some data showing discrepency between a harvard FP and a Touro CA FP living in the same area.

 

[cactusman]

Cohen?

I'm basically just working from google here (I'm not much of a statistician, so maybe I'm totally wrong), but I think he created the calculation for effect size that's generally used and came up with a system for classifying effect strength based on that number.

 

[deleted781806]

This exists for UG already. Mostly due to the fact that top UG institutions also tend to feed into professional programs. I am still waiting for some data showing discrepency between a harvard FP and a Touro CA FP living in the same area.

I hadn't thought about UG programs in my response, but I'm not surprised that such a correlation has been found. It would be very interesting to see how it holds up with medicals schools and residencies though, that would really help sort all this arguing out. I don't know if the same pattern would hold true with UG as with post-graduate professional programs (barring Law School)

 

[aldol16]

Cohen?

People on here like to defer to "authorities" on statistical analysis instead of thinking things through for themselves, so they defer to the "authority" as to what constitutes a "weak" vs. "moderate" vs. "strong" correlation.

 

[libertyyne]

People on here like to defer to "authorities" on statistical analysis instead of thinking things through for themselves, so they defer to the "authority" as to what constitutes a "weak" vs. "moderate" vs. "strong" correlation.

You seem bitter about this.

 

[cactusman]

People on here like to defer to "authorities" on statistical analysis instead of thinking things through for themselves, so they defer to the "authority" as to what constitutes a "weak" vs. "moderate" vs. "strong" correlation.

I don't know enough about statistics to trust my thought process. And some sources seemed to be a bit different, so I specified him (so there's confirmation bias I guess, but strong vs moderate is subjective anyway).

 

[efle]

It's sort of weird. The relation is strong enough I can understand why admissions would like high MCATs - having a median MCAT that is top 1-2% instead of top 15-20% would make for a significant step boost for the class even with only 40% strength.

At the same time it is weak enough that an individual can't really go in with much of an expectation about how their numbers will look for residency apps. Idk man.

 

[aldol16]

You seem bitter about this.

Well I like to teach my students to think about r^2 and what it really means so it's discouraging to see so many people so willing to defer to some established "authority" when it takes only a few steps to understand what variance is. Think for yourself (not you specifically but rather in general) - that's the only way to analyze data.

 

[cactusman]

Well I like to teach my students to think about r^2 and what it really means so it's discouraging to see so many people so willing to defer to some established "authority" when it takes only a few steps to understand what variance is. Think for yourself (not you specifically but rather in general) - that's the only way to analyze data.

While we're sort of on the topic, have you seen "Guess the Correlation?" It's the only online game worth playing.

 

[aldol16]

While we're sort of on the topic, have you seen "Guess the Correlation?" It's the only online game worth playing.

No, I haven't. What's it about?

 

[redsox93]

I wonder if there will be any difference with the new MCAT score

 

[Astra]

With the increased opportunities for networking and increased access to resources for academic support at top programs it makes sense that it would be easier for students at such institutions to achieve higher USMLE scores

What type of resources would top schools have access to so that students score higher on USMLE exams?

The materials used to prep for the exam are all from 3rd parties ( First Aid, Pathoma, Bros deck, Uworld etc).

 

[cactusman]

No, I haven't. What's it about?

It gives you a scatter plot and you guess the R value. So not very exciting, and it doesn't teach much, but it's surprisingly fun.

 

[deleted781806]

What type of resources would top schools have access to so that students score higher on USMLE exams?

The materials used to prep for the exam are all from 3rd parties ( First Aid, Pathoma, Bros deck, Uworld etc).

There might be a difference between having to buy those materials yourself, or having the institution provide them for free (no idea how the 3rd party materials work, to be honest). But, I have heard differences in how schools try to prep their students with structured coursework for USMLE prep, longer vs shorter time off before sitting for the exam, tutors provided by the institution, study plans written to fit individual students needs, etc. I'm just kind of spitballing, but I wouldn't be surprised if there are some differences between schools in how they approach USMLE prep. There's also always the hard-work factor for individual success, however.

 

[aalamruad]

It seems to have been 6 years since one of these posts were done. Someone stated that there isn't a correlation between MCAT and USMLE. This is a pretty big study that says there is. People who tend to do good on the SAT also do good on the MCAT also do good on the USMLE and also tend to do well in life. I'm kind of shocked that people even question this.

Undergraduate Institutional MCAT Scores as Predictors of USM... : Academic Medicine

I think the main thing people were questioning is that "doing well in life" = "making a lot of money"

It seems that students from top med schools tend to disproportionately go into competitive/high-paying specialties at great residency programs (just from looking through some match lists), but they also tend to disproportionately go into academic medicine post-residency/fellowship, which pays lower than PP. Those should counteract each other to some extent when it comes to salary, and it might be tough to determine which one plays a more significant role.

Rather than arguing that med students from top schools make more money than students from lower ranked schools, it seems like it would be more fair of you to simply argue that if students from top schools want to make a lot of money, they're better positioned to do so than students from lower ranked schools. 'twould avoid all the murkiness associated with incorporating the significance of academic medicine vs PP.

 

[cactusman]

I think the main thing people were questioning is that "doing well in life" = "making a lot of money"

It seems that students from top med schools tend to disproportionately go into competitive/high-paying specialties at great residency programs (just from looking through some match lists), but they also tend to disproportionately go into academic medicine post-residency/fellowship, which pays lower than PP. Those should counteract each other to some extent when it comes to salary, and it might be tough to determine which one plays a more significant role.

Rather than arguing that med students from top schools make more money than students from lower ranked schools, it seems like it would be more fair of you to simply argue that if students from top schools want to make a lot of money, they're better positioned to do so than students from lower ranked schools. 'twould avoid all the murkiness associated with incorporating the significance of academic medicine vs PP.

That wasn't the main thing I disagreed with (I do disagree, but I didn't really notice it originally).

I think this thread just shows that @Jalby made a lot of questionable claims and he's trying to make his overall argument seem better by defending more reasonable, dialed-back points that most people already accepted.

 

[aldol16]

What type of resources would top schools have access to so that students score higher on USMLE exams?

The materials used to prep for the exam are all from 3rd parties ( First Aid, Pathoma, Bros deck, Uworld etc).

Some schools have said they have access to NBME-written questions - it was never clear to me whether this was something they purchased or through an exclusive partnership with the NBME. Many top schools also have innovative curricula that allow students to get a lot of clinical exposure prior to taking the Step 1, which may or may not help improve scores (the schools that do this, understandably, argue that it does help improve scores).

 

[AnatomyGrey12]

and he's trying to make his overall argument seem better by defending more reasonable, dialed-back points that most people already accepted.

Exactly. The claim that "Higher MCAT = higher income" has now become "good test taker = Good test taker" and his previous argument of "top 5 medical school grad will make more money" shifted to become "higher quality residency will lead to better jobs".......

His changed arguments are all things that we all previously accepted.

 

[aldol16]

Exactly. The claim that "Higher MCAT = higher income" has now become "good test taker = Good test taker" and his previous argument of "top 5 medical school grad will make more money" shifted to become "higher quality residency will lead to better jobs".......

I don't think he's inconsistent at all - he agrees with the commonly-held notion that better residency = better jobs but he only takes it a step further, asserting that better med school = (greater chance of) better residency = better jobs. All one has to do is claim the transitive property to understand his conclusion. I think the better med school = (greater chance of) better residency is still open to debate and there is not a lot of data that exists for either side.

 

[cactusman]

I don't think he's inconsistent at all - he agrees with the commonly-held notion that better residency = better jobs but he only takes it a step further, asserting that better med school = (greater chance of) better residency = better jobs. All one has to do is claim the transitive property to understand his conclusion. I think the better med school = (greater chance of) better residency is still open to debate and there is not a lot of data that exists for either side.

The trouble is that Jalby says that

a) for the same person, the difference between mid-tier and top 5-10 income will probably be greater than $400k, because of residency placement, so the top school is almost universally a better choice over any amount of financial aid/scholarship money at a mid or low tier

b) he doesn't need evidence, because hes an attending who's just giving anecdotes (conveniently ignoring other attending posters who disagree with him, and weirdly certain that his anecdotes are part of a broader, very significant trend)

 

[aalamruad]

I don't think he's inconsistent at all - he agrees with the commonly-held notion that better residency = better jobs but he only takes it a step further, asserting that better med school = (greater chance of) better residency = better jobs. All one has to do is claim the transitive property to understand his conclusion. I think the better med school = (greater chance of) better residency is still open to debate and there is not a lot of data that exists for either side.

better job != higher paying job

 

[aldol16]

The trouble is that Jalby says that

a) for the same person, the difference between mid-tier and top 5-10 income will probably be greater than $400k, because of residency placement, so the top school is almost universally a better choice over any amount of financial aid/scholarship money at a mid or low tier

b) he doesn't need evidence, because hes an attending (conveniently ignoring other attending posters who disagree with him)

I'm not defending his views - I'm only saying that he's only taking a few steps beyond what is commonly accepted. That statistic could be true - it could not. I don't know personally.

 

[aldol16]

better job != higher paying job

That would depend on your definition of "better." Many people do in fact (shocker!) equate higher-paying with better.

 

[RenegadeOkapi]

Um, I'm not sure. People's work ethic tend to change, and from high school to undergraduate to med school, there is a pretty big room for change.
I'll be off to med school in the fall so I don't have Step 1 scores, but my scores thus far might be an outlier:

ACT: 36
SAT:2380
MCAT: 513

Not the biggest correlation.

 

[Astra]

That would depend on your definition of "better." Many people do in fact (shocker!) equate higher-paying with better.

I'd personally want that sweet spot where I can have a life outside of medicine and still be payed well.

What's the use of spending an entire life making a fortune if you never enjoy life?

 

[aalamruad]

That would depend on your definition of "better." Many people do in fact (shocker!) equate higher-paying with better.

That's my point, Jalby's mistake is rooted in the belief that everyone agrees with his definition of better jobs being those that pay well, when a lot of people value academic gigs more than he seems to.

 

[cactusman]

I'm not defending his views - I'm only saying that he's only taking a few steps beyond what is commonly accepted. That statistic could be true - it could not. I don't know personally.

But I do think this thread is misrepresenting his argument by reducing it just to scores and ignoring the very important "strong" part of his statement about correlation, and correlation with income.

That's my point, Jalby's mistake is rooted in the belief that everyone agrees with his definition of better jobs being those that pay well, when a lot of people value academic gigs more than he seems to.

OTOH, he's probably right about how to choose if we're thinking academia rather than money.

 

[AnatomyGrey12]

That would depend on your definition of "better." Many people do in fact (shocker!) equate higher-paying with better.

Even this I am not sure I believe it. Some of the highest paying jobs out there are the rural positions that have a lot of trouble finding people that want them. It's okay though, just means more future job options for little old me.

 

[Goro]

I don't have the citations handy, but there are a number of them that state that preclinical GPAs are the best predictors of Step I and COMLEX I scores.

It seems to have been 6 years since one of these posts were done. Someone stated that there isn't a correlation between MCAT and USMLE. This is a pretty big study that says there is. People who tend to do good on the SAT also do good on the MCAT also do good on the USMLE and also tend to do well in life. I'm kind of shocked that people even question this.

Undergraduate Institutional MCAT Scores as Predictors of USM... : Academic Medicine

 

[libertyyne]

I don't think he's inconsistent at all - he agrees with the commonly-held notion that better residency = better jobs but he only takes it a step further, asserting that better med school = (greater chance of) better residency = better jobs. All one has to do is claim the transitive property to understand his conclusion. I think the better med school = (greater chance of) better residency is still open to debate and there is not a lot of data that exists for either side.

The largest piece of his argument that is problematic imho is that he asserts that a "better residency" placement will lead to higher paying job in the same specialty. There are plenty of MGH trained Family practice doctors that I am willing to bet that do not make more then their peers that were trained in family practice elsewhere. Medicare, and private payors do not have a modifier for "better residency" when reimbursing physicians for the same service. Most jobs dont care where you went to residency as long as you are board certified.

The second confounder is that people going to top medical schools are more likely to have on average higher mcat scores compared to mcat scores on average of matriculants else where. There is no surprise that these folks are more likely to have higher board scores leading to better specialty placement. Then how do you tease apart the placement / matchlist discrepancies?

 

[mwsapphire]

I don't have the citations handy, but there are a number of them that state that preclinical GPAs are the best predictors of Step I and COMLEX I scores.

What if you have P/F?
HHHMmmmm?

 

[Goro]

Just because the students have P\F, it doesn't mean the Faculty lack the % grade info.

What if you have P/F?
HHHMmmmm?

 

[Jalby]

Jalby could be right for reasons besides standardized test performance. Students at Harvard likely have more resources and networking opportunities to strengthen long-term career goals compared to students at state schools.

I've said this exact thing many many many times. If you go to a top five medical school than it is likely that the residency director at a different school knows or trained or was taught by one of your mentors and can call them for an honest opinion which gives you a HUGE step up. I've also stated at my mid-level radiology practice that the most important thing in deciding who we hire is if we have someone we trust recommending them.

 

[Jalby]

I dont think, this is a far fetched argument set forth by @Jalby. What I sincerely doubt is that there is a large income differential between MD graduates of state X school and Harvard provided both of them completed the same specialty residency and reside in the same area.

I didn't know tuition had increased to $60k nowadays. In my days $55k was the his best and USC was $22k cheaper. So with interest you need to pay off about $500k which means you need a $25k difference over a 20 year career.

 

[mistafab]

I'd just like to add to your discussions that just because the r is .26 or .89 or any number you through out there, the most important factor is the p-value of the correlation in question. If the p value is exceptionally small, that means that the relationship is very likely a real and true thing.

Make sense? So the discussion of saying "I don't think .3 is meaningful" does not mean that you can just deny the relationship. I haven't seen a single p-value at all in this thread. Smoking, last I checked, had an r value of around .2-.3 for lung cancer. Make sense? The relationship that smoking has with lung cancer is extremely strong, even if the effect is "small," it is still considered very important because of the strong evidence that the relationship is indeed there.

For this mcat corr'd with step score corr'd with what have you talk - the relationships are probably all there. You guys are just debating the effect of the correlation really. To say that because the effect is small so the relationship does not exist is just a fallacy.

 

[efle]

I'd just like to add to your discussions that just because the r is .26 or .89 or any number you through out there, the most important factor is the p-value of the correlation in question. If the p value is exceptionally small, that means that the relationship is very likely a real and true thing.

Make sense? So the discussion of saying "I don't think .3 is meaningful" does not mean that you can just deny the relationship. I haven't seen a single p-value at all in this thread. Smoking, last I checked, had an r value of around .2-.3 for lung cancer. Make sense? The relationship that smoking has with lung cancer is extremely strong, even if the effect is "small," it is still considered very important because of the strong evidence that the relationship is indeed there.

For this mcat corr'd with step score corr'd with what have you talk - the relationships are probably all there. You guys are just debating the effect of the correlation really. To say that because the effect is small so the relationship does not exist is just a fallacy.

The ranges in parentheses in a table like this are the 95% CI... they are far above zero (.50-.67 range for the overall MCAT) so it is quite significant.

Make sense?

 

[aldol16]

Even this I am not sure I believe it. Some of the highest paying jobs out there are the rural positions that have a lot of trouble finding people that want them. It's okay though, just means more future job options for little old me.

And there will be people who believe that those rural positions are the best jobs for the sole fact that they are being paid the most. That's fine with me - different people have different definitions of "best" and who am I to judge what floats your boat?

 

[RogueUnicorn]

my MCAT was baddddd, my step 1 is a 232. The difference? I was lazy and didn't work hard for MCAT at all.

Dude a 232 is like just a bit above average. Just saying.

 

[libertyyne]

I didn't know tuition had increased to $60k nowadays. In my days $55k was the his best and USC was $22k cheaper. So with interest you need to pay off about $500k which means you need a $25k difference over a 20 year career.

Yes, you have shown us that calculation before, the question is where are you getting this income differential data where doctors of x specialty are getting paid 25K more compared to their peers who did x residency at podunk U.

 

[Lawpy]

Dude a 232 is like just a bit above average. Just saying.

230 is roughly the new Step 1 average? Surprised to see such a steep score creep, since I read in only recently old threads (back in 2010 i think) that Step 1 average was around 220. I don't have the data available so I could be wrong. It just seems to me 230 is a bit too high for a Step 1 average.

 

[libertyyne]

230 is roughly the new Step 1 average? Surprised to see such a steep score creep, since I read in only recently old threads (back in 2010 i think) that Step 1 average was around 220. I don't have the data available so I could be wrong. It just seems to me 230 is a bit too high for a Step 1 average.

With our luck they will introduce the new STEP 1 when we get ready to take it, like the MCAT and rebalance the scales. 230 is roughly a 500 in terms of distribution so a 26 on the old mcat.

No correlation with me. MCAT 28 and Step 1 261.

Where did you go to school?

 

[aldol16]

I'd just like to add to your discussions that just because the r is .26 or .89 or any number you through out there, the most important factor is the p-value of the correlation in question. If the p value is exceptionally small, that means that the relationship is very likely a real and true thing.

Make sense? So the discussion of saying "I don't think .3 is meaningful" does not mean that you can just deny the relationship. I haven't seen a single p-value at all in this thread. Smoking, last I checked, had an r value of around .2-.3 for lung cancer. Make sense? The relationship that smoking has with lung cancer is extremely strong, even if the effect is "small," it is still considered very important because of the strong evidence that the relationship is indeed there.

You're talking about two very different things here. One is whether an effect is there and another is the strength of that effect. If there is a strong effect present, the probability of that effect being not real as measured by p-value becomes smaller. Think about what p-value is. p-value measures the probability that the difference between your p-value and the mean is just caused by randomness. So the more clustered your measurements, the more likely the difference will be real and thus have a small p-value. More clustered measurements = smaller variance.

But you can also run into the opposite error here, i.e. your p-value being sufficiently small but it's actually not significant in the real world. So say you measure weight gain as a function of how much junk food you eat. And you find that each bag of chips you eat increases your weight by 0.00000001 pounds. And this effect is so startling that there's very little variance - that is, if you eat that one extra bag of chips, you will almost certainly increase your weight by exactly 0.00000001 pounds. And since you have almost no variance, let's say that the p-value is also very small - 0.001 or something like that. So it's both real and you have a really good r^2. But does it mean anything? Even though that bag of chips will certainly or almost certainly increase your weight by that miniscule amount, the effect doesn't matter. It's so small on the real scale that we can dismiss it, even though you have a small p-value and high r^2.

 

[Major procrastinator]

Dude a 232 is like just a bit above average. Just saying.

what did you get?

 

[Lawpy]

With our luck they will introduce the new STEP 1 when we get ready to take it, like the MCAT and rebalance the scales.

I really feel bad for MD/PhD students. From the way Step 1 score creep is accelerating, their competitive Step 1 scores taken at end of M2 may actually become significantly less competitive (or worse, below average!) by the time they apply for residency ~5-6 years later.

 

[Major procrastinator]

With our luck they will introduce the new STEP 1 when we get ready to take it, like the MCAT and rebalance the scales. 230 is roughly a 500 in terms of distribution so a 26 on the old mcat.


Where did you go to school?

considering havards average is a 240, a 232 is not bad.

you premeds have no idea what your talking about, welcome to the real world when your are no longer obnxious dinguses.

why did I make the mistake of posting here

 

[RogueUnicorn]

230 is roughly the new Step 1 average? Surprised to see such a steep score creep, since I read in only recently old threads (back in 2010 i think) that Step 1 average was around 220. I don't have the data available so I could be wrong. It just seems to me 230 is a bit too high for a Step 1 average.

It's somewhere in the middle 220s with a stdev of something like 10-15

what did you get?

The middle number was a 6.
Typo, it was a 5

 

[Major procrastinator]

With our luck they will introduce the new STEP 1 when we get ready to take it, like the MCAT and rebalance the scales. 230 is roughly a 500 in terms of distribution so a 26 on the old mcat.


Where did you go to school?

lol a 26? your math is a joke

 

[Major procrastinator]

230 is roughly the new Step 1 average? Surprised to see such a steep score creep, since I read in only recently old threads (back in 2010 i think) that Step 1 average was around 220. I don't have the data available so I could be wrong. It just seems to me 230 is a bit too high for a Step 1 average.

when it comes to USA and canadian grads 229 +/- 20 is the mean. if you take into account all students that take step 1 it drops significantly. a 232 is in the 76th percentile overall.

 

[Lawpy]

I'd just like to add to your discussions that just because the r is .26 or .89 or any number you through out there, the most important factor is the p-value of the correlation in question. If the p value is exceptionally small, that means that the relationship is very likely a real and true thing.

Make sense? So the discussion of saying "I don't think .3 is meaningful" does not mean that you can just deny the relationship. I haven't seen a single p-value at all in this thread. Smoking, last I checked, had an r value of around .2-.3 for lung cancer. Make sense? The relationship that smoking has with lung cancer is extremely strong, even if the effect is "small," it is still considered very important because of the strong evidence that the relationship is indeed there.

For this mcat corr'd with step score corr'd with what have you talk - the relationships are probably all there. You guys are just debating the effect of the correlation really. To say that because the effect is small so the relationship does not exist is just a fallacy.

The ranges in parentheses in a table like this are the 95% CI... they are far above zero (.50-.67 range for the overall MCAT) so it is quite significant.

Make sense?

You're talking about two very different things here. One is whether an effect is there and another is the strength of that effect. If there is a strong effect present, the probability of that effect being not real as measured by p-value becomes smaller. Think about what p-value is. p-value measures the probability that the difference between your p-value and the mean is just caused by randomness. So the more clustered your measurements, the more likely the difference will be real and thus have a small p-value. More clustered measurements = smaller variance.

But you can also run into the opposite error here, i.e. your p-value being sufficiently small but it's actually not significant in the real world. So say you measure weight gain as a function of how much junk food you eat. And you find that each bag of chips you eat increases your weight by 0.00000001 pounds. And this effect is so startling that there's very little variance - that is, if you eat that one extra bag of chips, you will almost certainly increase your weight by exactly 0.00000001 pounds. And since you have almost no variance, let's say that the p-value is also very small - 0.001 or something like that. So it's both real and you have a really good r^2. But does it mean anything? Even though that bag of chips will certainly or almost certainly increase your weight by that miniscule amount, the effect doesn't matter. It's so small on the real scale that we can dismiss it, even though you have a small p-value and high r^2.

>tries to educate SDNers about statistics by mentioning p-values
>ends up being educated in the process

Well done SDN.

 

[libertyyne]

lol a 26? your math is a joke

I dont know why you are getting offended.