[Medical Teacher] Using Casper to predict professionalism behaviors

[Mr.Smile12]

Advertisement - Members don't see this ad

Beyond admissions: Using situational judgment test scores to predict learner performance throughout undergraduate medical education (University of Michigan Medical School)

Situational judgment tests (SJTs) have been introduced in medical school admissions across multiple national contexts over the past three decades. However, ongoing research is needed to understand SJTs’ role in predicting outcomes across undergraduate medical education (UME), particularly in the context of institution-specific approaches to medical student assessment. This study investigated the role of the Casper SJT for understanding student performance in professionalism, communication and self-awareness at three key timepoints in admissions and UME: interview day scores, the preclinical phase, and the clinical phase.

Beyond admissions: Using situational judgment test scores to predict learner performance throughout undergraduate medical education

Situational judgment tests (SJTs) have been introduced in medical school admissions across multiple national contexts over the past three decades. However, ongoing research is needed to understand ...

www.tandfonline.com

 

[Wistfulness]

It suits my narrative, therefore the study is super valid and high-quality. Once again the constructs this exam claims to measured have not been well-defined, validated or litigated... why should we take any result... positively or negatively seriously?

Second, including the Casper score in the communication domain model improved its explanatory power by 3.1% the Casper score could add some practical value to assessment of learners’ communication skills during the admissions process.

3%? You know what happens to index funds that bring 3% returns? Dropped from my holdings.

Similarly, including the Casper score improved the model of outcomes related to self-awareness.

Then if you scroll up and read the section about self-awareness outcomes they say that the model was not statistically significant.

I just want a (deeply unethical) Mr. Beast-style social experiment where admissions committee members are forced take the exam, and we tie something of value to Casper scores. They get their pay docked if they score below the average and a raise if they score above the average. The amount will be the average cost of applying to medical school which is a measly 10K, hardly anything to get your knickers in a twist over. We'll do a survey about their thoughts on the validity and fairness of the exam before and after the results are returned.

We won't actually give them their scores back, but instead a score most counter to their position in the before survey. Rewards will be granted to those who remained consistent.

 

[Mr.Smile12]

I just want a (deeply unethical) Mr. Beast-style social experiment where admissions committee members are forced take the exam, and we tie something of value to Casper scores. They get their pay docked if they score below the average and a raise if they score above the average. The amount will be the average cost of applying to medical school which is a measly 10K, hardly anything to get your knickers in a twist over. We'll do a survey about their thoughts on the validity and fairness of the exam before and after the results are returned.

We won't actually give them their scores back, but instead a score most counter to their position in the before survey. Rewards will be granted to those who remained consistent.

That would be very social-medial clickbait-ish. 🙂 Have all these physician-leaders go back and take the MCAT. 🙂

A little seriously though, I would not recommend Casper without the chance for my adcoms to take it themselves and figure out if they would accept themselves into the entering class. Unfortunately that idea was shot down. (On the other hand, when MMI's became popular, giving faculty a test run as examinees was an important step to adopting or not.) PREview is already skewed because the question-writers are also med school faculty or admins.

I like to document the negative outcomes, especially when they are run independently from the testing companies. There are significant curricular differences in the value of a third-party-written exam on all health professional curricula and cultures, and this study by UMMS and another by Icahn argue that point. It doesn't work for everyone like it would at McGill (where it was created).