Declining validity for MCAT! Interesting piece

[paranoid99]

---

Members don't see this ad.

http://www.insidehighered.com/news/2010/05/27/qt/declining_validity_for_mcat

Declining Validity for MCAT



The Medical College Admission Test has generally been successful in predicting students’ success during medical school and residency training and on licensing exams, according to a study published in Academic Medicine. However, as the test has been changed over the years, validity has declined on part of the national medical licensing exam, the study found. Further, it found no correlation between MCAT scores and students’ clinical competence as residents because the “validity coefficients were either nonsignificant or practically negligible.”

 

[marele86]

http://journals.lww.com/academicmed...ive_Validity_of_Three_Versions_of_the.20.aspx

(The full article). For the record, looks like it's only talking about Step 2 validity declining.

"No significant improvement in validity coefficients was observed for performance in medical school or residency. Validity coefficients for all three versions of the MCAT in predicting Part I/Step 1 remained stable (in the mid-0.40s, P < .01). A systematic decline was observed in the validity coefficients of the MCAT versions in predicting Part II/Step 2. It started at 0.47 for the pre-1978 version, decreased to between 0.42 and 0.40 for the 1978–1991 versions, and to 0.37 for the post-1991 version. Validity coefficients for the MCAT versions in predicting Part III/Step 3 remained near 0.30."

Interesting article, though. 👍 Didn't even know the AAMC had a journal until I went to look this up.

 

[Lokhtar]

What was it asking pre-1978 that would have good predictive value for step II performance?

 

[marele86]

What was it asking pre-1978 that would have good predictive value for step II performance?

"Subtest scores of the MCAT were used as predictors. For Group 1 (matriculants between 1970 and 1977), scores on four subtests of the pre-1978 version of the MCAT (Science Achievement, General Information, Quantitative Ability, and Verbal Ability) were used."

I think the table shows that Science Achievement and Gen. Info were the best predictors for Step 2 pre-1978 - the images aren't loading right on my computer for some reason.

But man, I'm glad we don't have a General Information MCAT section anymore. Aside from being all kinds of biased, I bet it would've been a pain to study for.

 

[Lokhtar]

But man, I'm glad we don't have a General Information MCAT section anymore. Aside from being all kinds of biased, I bet it would've been a pain to study for.

Q1. When you drink Chateau Mouton Rothschild Pauillac 1986, which combination of flavors is the most apparent?

A. Chocolate, Raspberry, Spices
B. Chocolate, Strawberry, Vanilla
C. Vanilla, Raspberry, Isabella
D. Isabella, Strawberry, Spices

How can general information possibly be biased? 😛

 

[thesauce]

http://journals.lww.com/academicmed...ive_Validity_of_Three_Versions_of_the.20.aspx

(The full article). For the record, looks like it's only talking about Step 2 validity declining.

"No significant improvement in validity coefficients was observed for performance in medical school or residency. Validity coefficients for all three versions of the MCAT in predicting Part I/Step 1 remained stable (in the mid-0.40s, P < .01). A systematic decline was observed in the validity coefficients of the MCAT versions in predicting Part II/Step 2. It started at 0.47 for the pre-1978 version, decreased to between 0.42 and 0.40 for the 1978–1991 versions, and to 0.37 for the post-1991 version. Validity coefficients for the MCAT versions in predicting Part III/Step 3 remained near 0.30."

Interesting article, though. 👍 Didn't even know the AAMC had a journal until I went to look this up.

Nobody takes step 2 seriously anyway