Some Residency Programs Now Requiring CASPer for 2021 Match

[sunshinefl]

It was sad long before I posted. Trying to pass comments like this off as humor is sad. The fact that so many of you support the comments is sadder. And the fact that the poster doesn’t even see his comments as being arrogant or uninformed and thinks that it’s okay to be unprofessional if you frame it as humor is even sadder still.

The fact you keep using “unprofessional” repeatedly as your main argument only confirms you are in admin.

---

Members don't see this ad.

 

[slowthai]

Spoke truth about some admin? Yes I’m sure that most admins have no better motivation behind their decisions than making poor medical students’ and residents’ lives hell.
If they truly had malicious intentions, your life would be much more hellish.

And you honestly don’t think that comments like the one made about admins contribute to the toxic culture of medicine? I suppose you also don’t think any physicians or residents are ever disrespectful of other health professionals.

I’m sorry that my views make you sad, but I truly am a resident. I’m not an admin in any medical schools or health care systems and never have been one. I suppose it’s only fitting that your views make me equally sad, because until those views change the culture of medicine will continue to be as toxic as it is today.

Calling out BS isn't toxic. What's toxic is confidently giving terrible career advice, lying, and being ridiculously petty. These are all things admins excel at.

 

[R1.5]

You're new here, aren't you?

May I suggest that the rest of you keep the thread on topic?

Calling out BS isn't toxic. What's toxic is confidently giving terrible career advice, lying, and being ridiculously petty. These are all things admins excel at.

Calling out BS in a productive way isn’t toxic. Bitching about admins on a public forum, making sweeping generalizations about them as a group and with zero firsthand knowledge of them or their actual motivations, and persistently accusing someone that doesn’t agree with your approach or assumptions is toxic. The fact that you can’t see it doesn’t change it. If you instead approached the admin responsible for the decision and provided your perspective in a respectful way that would allow them to explain their decision and rationale behind it, I would consider that non-toxic and collegial.

I’m finished engaging with this group about the subject. Initially I hoped that I might encourage introspection about how your opinion does not equal fact and how slandering an entire field of work is unprofessional. It is now clear that your mind is closed to any possibility that you have done any wrong here. So rather than belabor the point I will just hope that one day you are able to see what you refuse to see now.

 

[trapperjohn4077]

It is now clear that your mind is closed to any possibility that you have done any wrong here.

We aren't closed-minded: we disagree. You come in here with your toxicity, throwing around the "P" word at anyone who dares call into question the decisions of administration. Guess what, you have no power here. "Professionalism" has been used to control and manipulate students and residents alike, by holding the sword of damacles over their heads. We have no power to "approach the admin" in any meaningful way, as we risk being labeled as a difficult student. An admin can tank someone's career just by saying "unprofessional" - which by the way is often a code word for "disobedient". In the worst case scenario, it can mean "lacking in deference", which is the way you're using it now.

Btw... people should question and criticize authority, and remember that humor is a "mature" defense mechanism.

 

[0ncogene]

We had to do a test similar to CASper this during residency, so the program can evaluate our intrinsic qualities and weaknesses. It was useless, despite their website claiming evidence supporting accuracy of test and ways improve problem solving, communication, and professionalism. Thank goodness our PD thought it was useless too.

And yes admins have reasons why they implement certain things, but they are never in the interest of the medical students, residents, or even attending physicians. Their priorities are to the medical school or hospital that employ them. They are bean counters. They don't care if they make you jump through hundreds of additional hoops and hurdles to make their job easier. Unfortunately, as medical students, residents, and employees of the hospital, there is not much you can do. You just have to be aware of this and make sure you participate in committee meetings and take leadership roles in the future to voice your concerns, because this is where policies are made and changed.

 

[WheezyBaby]

Unsolicited opinion: I can say with 100% certainty that zero thought went into this. You know, as per usual with admin. They just like to do stupid stuff for stupid stuff's sake.

Admin 1: Step 1's going pass/fail. What are we gonna do now?
Admin 2: Uhh, I hear med schools are doing this casper thing?
Admin 1: Hmm, does it work?
Admin 2: No idea
Admin 1: Great, let's do it

You know when you say admin here, what you mean is "program director", right? These aren't people distanced from direct patient care or resident training. I'm not well versed enough in the literature to say SJT's are the way to go for improving candidate selection, but there is at least some literature there

https://www.researchgate.net/publication/341507860_Improving_Our_Ability_to_Predict_Resident_Applicant_Performance_Validity_Evidence_for_a_Situational_Judgment_Test

Predicting Performance of First-Year Residents: Correlations Between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study | Request PDF

Request PDF | Predicting Performance of First-Year Residents: Correlations Between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study | Purpose: To determine whether scores on structured interview (SI) questions designed to measure noncognitive...

www.researchgate.net

My inclination is that if SJT's become standard, SJT-directed prep will more or less invalidate it, and it'll become another go through the motions money grab like CS. Regardless, the people implementing this are those invested in improving the selection of residents they'll be working closely with, not shadow figures in a c suite.

Also, I haven't seen @Med Ed post recently but don't remember them being so snarky, I like it

 

[Matthew9Thirtyfive]

https://www.researchgate.net/publication/341507860_Improving_Our_Ability_to_Predict_Resident_Applicant_Performance_Validity_Evidence_for_a_Situational_Judgment_Test

Uh, did you see the Pearson coefficients? Not sure I’d be using that as evidence of a good correlation.

 

[WheezyBaby]

Uh, did you see the Pearson coefficients? Not sure I’d be using that as evidence of a good correlation.

What do we have in place that already has an excellent correlation with all acgme core competencies? Not to even get into how difficult measuring the core competencies themselves is

 

[slowthai]

You know when you say admin here, what you mean is "program director", right? These aren't people distanced from direct patient care or resident training. I'm not well versed enough in the literature to say SJT's are the way to go for improving candidate selection, but there is at least some literature there

https://www.researchgate.net/publication/341507860_Improving_Our_Ability_to_Predict_Resident_Applicant_Performance_Validity_Evidence_for_a_Situational_Judgment_Test

Predicting Performance of First-Year Residents: Correlations Between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study | Request PDF

Request PDF | Predicting Performance of First-Year Residents: Correlations Between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study | Purpose: To determine whether scores on structured interview (SI) questions designed to measure noncognitive...

www.researchgate.net

My inclination is that if SJT's become standard, SJT-directed prep will more or less invalidate it, and it'll become another go through the motions money grab like CS. Regardless, the people implementing this are those invested in improving the selection of residents they'll be working closely with, not shadow figures in a c suite.

Also, I haven't seen @Med Ed post recently but don't remember them being so snarky, I like it

In the context in which I was using it, admin is just a general term for the group of people most involved with resident selection, which includes PDs.

The fact that they're invested in resident selection doesn't prevent them from making poorly thought out decisions. I mean, some programs are using freaking midlevels to interview applicants. It's totally insane.

 

[WheezyBaby]

In the context in which I was using it, admin is just a general term for the group of people most involved with resident selection, which includes PDs.

The fact that they're invested in resident selection doesn't prevent them from making poorly thought out decisions. I mean, some programs are using freaking midlevels to interview applicants. It's totally insane.

The people most involved with resident selection = PD, aPD's, interviewing faculty, residents, and department chair. With the exception potentially of the department chair, these aren't the people typically classified pejoratively as "admin".

 

[Matthew9Thirtyfive]

What do we have in place that already has an excellent correlation with all acgme core competencies? Not to even get into how difficult measuring the core competencies themselves is

I mean adding things with terrible correlation just to have something isn’t the answer.

 

[WheezyBaby]

I mean adding things with terrible correlation just to have something isn’t the answer.

Any incremental validity has some worth. You're not going to find anything with a 0.7 for this type of work. Everything we have sucks. Regardless, I'm not arguing for SJT's, I'm arguing against the notion these are admin removed from the training process choosing to implement bull**** for the sake of bull****

 

[Matthew9Thirtyfive]

Any incremental validity has some worth. You're not going to find anything with a 0.7 for this type of work. Everything we have sucks. Regardless, I'm not arguing for SJT's, I'm arguing against the notion these are admin removed from the training process choosing to implement bull**** for the sake of bull****

I mean id settle for 0.5. An r=0.17? Come on, son. That’s completely worthless and is nothing more than a hoop to weed out applicants.

 

[MDMechEngr]

Three of 5 MD schools that accepted me last year require CASPer. If residency requires it, bring it on. Love the Admin at my school.

 

[Steve_Zissou]

Three of 5 MD schools that accepted me last year require CASPer. If residency requires it, bring it on. Love the Admin at my school.

Are you a first year?

 

[Lawpy]

Casper for residency is really dumb. Like clinical years and MSPEs already assess personal attributes and have far better value than an SJT with unrealistic and cliched scenarios. This is the dumbest hoop to add and a complete sellout to 3rd party businesses

 

[Lawpy]

Also why are US MDs and DOs punished with this CS-type situation when the target applicants should be IMGs/FMGs whose educational quality and language barriers are highly variable?

 

[FrontierPsychiatry]

Cheers to the crybabys calling for p/f step, this is what you guys get now

I mean as somebody who took scored step, and thinks the move to p/f was justified given the original purpose of the exam, this doesn't seem like a big deal, a minor inconvenience. Granted I don't have to take it, but of all the things in medical school and residency it really does seem quite insignificant.

 

[Med Ed]

I was curious if CASPer was still $10, and when I went to fake schedule a GME test I got the following (emphasis added):

"There is no fee to take the Casper test as part of the GME pilot."

 

[Lawpy]

I mean as somebody who took scored step, and thinks the move to p/f was justified given the original purpose of the exam, this doesn't seem like a big deal, a minor inconvenience. Granted I don't have to take it, but of all the things in medical school and residency it really does seem quite insignificant.

It's a waste! Why are US MDs and DOs forced to take Casper when there are far better metrics to assess their personal/ethics/morality etc.?

Seeing that a lot of med school admissions is requiring Casper already, why are MS4s forced to take that stupid test again?

 

[Med Ed]

It's a waste! Why are US MDs and DOs forced to take Casper when there are far better metrics to assess their personal/ethics/morality etc.?

Seeing that a lot of med school admissions is requiring Casper already, why are MS4s forced to take that stupid test again?

It's a pilot. Nine residency programs out of approximately 5,000.

Aren't there bigger things to worry about? Can't we just go back to complaining about nurses with PhD's?

 

[Lawpy]

It's a pilot. Nine residency programs out of approximately 5,000.

Aren't there bigger things to worry about? Can't we just go back to complaining about nurses with PhD's?

Iirc didn't it start as a pilot for med admissions before being adopted universally? What's to say the same wouldn't happen at residency app level?

If Casper is going to be here to stay, at least require it for IMGs/FMGs because requiring it twice for US MDs and DOs is bad

 

[Med Ed]

Iirc didn't it start as a pilot for med admissions before being adopted universally? What's to say the same wouldn't happen at residency app level?

If Casper is going to be here to stay, at least require it for IMGs/FMGs because requiring it twice for US MDs and DOs is bad

CASPer has not been adopted universally. It is required by less than one third of US allopathic medical schools. And it was piloted; researchers at McMaster published its correlation with MMI scores in 2009. You can find the who spiel about its validity and reliability here.

I'm going to wait and see what happens with the GME pilot before forming an opinion. Perhaps you should try that, too.

 

[Med Ed]

Simple answer: Boycott applying to programs that require CASPER. Snuff this flame out before it spreads.

If all that does is reduce the applicant pool at those programs without negatively impacting their respective matches then this approach will backfire spectacularly.

 

[Lawpy]

CASPer has not been adopted universally. It is required by less than one third of US allopathic medical schools. And it was piloted; researchers at McMaster published its correlation with MMI scores in 2009. You can find the who spiel about its validity and reliability here.

I'm going to wait and see what happens with the GME pilot before forming an opinion. Perhaps you should try that, too.

Based on your experience, what admissions pilot project hasn't become a requirement in future years?

 

[Matthew9Thirtyfive]

It's a pilot. Nine residency programs out of approximately 5,000.

Aren't there bigger things to worry about? Can't we just go back to complaining about nurses with PhD's?

If they actually had PhDs, they’d probably get less flak.

 

[Matthew9Thirtyfive]

CASPer has not been adopted universally. It is required by less than one third of US allopathic medical schools. And it was piloted; researchers at McMaster published its correlation with MMI scores in 2009. You can find the who spiel about its validity and reliability here.

I'm going to wait and see what happens with the GME pilot before forming an opinion. Perhaps you should try that, too.

I took Casper when I applied to med school. It was a complete waste of time and $10.

 

[Goro]

Based on your experience, what admissions pilot project hasn't become a requirement in future years?

Nice moving the goal posts.

 

[Lawpy]

Nice moving the goal posts.

No it's part of my main point that what starts as pilot becomes utilized by a lot of programs in following years. Which is why i'm slamming the test now rather than waiting to see how the pilot turns out.

If casper is a one time, experimental pilot that's not utilized in the future, i'll admit i overreacted (although it sucks for people applying to those programs having to take it now)

 

[Med Ed]

If they actually had PhDs, they’d probably get less flak.

I feel like there has been a thread or two about this topic...

 

[Med Ed]

Based on your experience, what admissions pilot project hasn't become a requirement in future years?

I don't have any experience that addresses what I think you're getting at.

Technically the only universal requirements for medical school admission are an application and interview. The MCAT is widely used, but some niche programs (e.g. Flexmed) do not require it. Most schools rely on similar pre-reqs, but some places have shed them and basically rely on the MCAT. The MMI was created at McMaster to provide some standardization over unstructured interviews, which don't seem to have much reliability. MMIs have spread but are by no means universal, and some high profile med ed people hate the MMI. Much of the reason the MMI gained traction is because it does not need faculty to run, so it's far more cost effective than the traditional approach.

Not many places publish or otherwise share failed pilots. A quick lit search did turn up this British pilot study that sought to use several tests of non-cognitive traits in the selection process. Their conclusion:

Our findings are consistent with previously published research. The tests had a very limited ability to predict undergraduate academic performance, though further research on identifying narcissism in medical students may be warranted. However, the validity of such self-report tools in high-stakes settings may be affected, making such instruments unlikely to add value within the selection process.

Could CASPer overtake GME in spite of having limited or no defined utility? Possibly. But I don't think that outcome is by any means inevitable. And I don't think complaining about it on SDN will have any effect whatsoever.

 

[trapperjohn4077]

I'm going to wait and see what happens with the GME pilot before forming an opinion. Perhaps you should try that, too.

Those of us who had to take CASPER for medical school interviews know what a horrible system it is already. We don't need to wait to hear what some ivory tower admins think about it. Perhaps you should consider taking a stand against it before it becomes ingrained in the system. It's easier to stop something in its tracks than to remove an embedded practice.

 

[Med Ed]

Those of us who had to take CASPER for medical school interviews know what a horrible system it is already. We don't need to wait to hear what some ivory tower admins think about it.

Get back to me after your first 10-year board recert.

Perhaps you should consider taking a stand against it before it becomes ingrained in the system. It's easier to stop something in its tracks than to remove an embedded practice.

Stand all you want, but I don't make decisions without information. Besides, I'm not even in GME, and PD's don't care what either of us thinks or wants.

 

[TelemarketingEnigma]

Based on your experience, what admissions pilot project hasn't become a requirement in future years?

The SVI is one example i can think of - though it was a requirement in EM for a bit, it ultimately got dropped.

 

[Matthew9Thirtyfive]

Get back to me after your first 10-year board recert.

So...what does this mean?

Stand all you want, but I don't make decisions without information. Besides, I'm not even in GME, and PD's don't care what either of us thinks or wants.

We have information. It at best correlated with an r=0.21. That is complete trash.

 

[Med Ed]

So...what does this mean?

It means I have literally seen people with their intestines partially extruded from their abdomens who have complained less than you guys are about a $12 test that takes less than 2 hours. After finishing Step 1, Step 2 CK, Step 2 CS, Step 3, your specialty boards (+/- oral exam), possibly your subspecialty boards, and then getting fleeced by MOC for the rest of your careers, CASPer will seem like a youthful opium dream.

We have information. It at best correlated with an r=0.21. That is complete trash.

If you are referring to the paper linked by @WheezyBaby above, that study doesn't even involve CASPer, but rather a homebrew SJT.

I also would not be so quick to judge a test based solely on the r. I'll give you an example. At my institution the MCAT is a poor predictor of preclinical performance as long as the scores are above a certain threshold. Go below that threshold land the MCAT quickly becomes a much stronger predictor of academic difficulties. We see a similar phenomenon with preclinical performance and USMLE scores, with the former being most predictive at the extremes, and much weaker in the middle (which comprises most of the students). Even if these associations are not super strong they are still useful, as they help flag individuals who are at higher risk of needing intervention.

You also have to look at the incremental validity of the test relative to its cost (both monetary and time). A quick, inexpensive test does not necessarily need to add a huge amount of value to a multivariate predictive model in order to justify its use.

To that end, it would be helpful to know what question(s) the people behind the GME pilot are actually asking, and what they are hoping to gain from trying CASPer. But neither of us are privy to that, so we are missing important information.

Look, there are plenty of problems to be had. MMIs all have systematic biases, unstructured interviews don't correlate with anything, but few question them because they are a familiar and time-honored way of doing things. Step scores have been misused for years. At the root of all this evil, however, is the phenomenon of over-application. Since no one wants to try capping residency applications, I think it will take the development of widespread preference signaling to bring this whole mess somewhat under control, and mitigate the need for more GME pilots.

 

[Matthew9Thirtyfive]

It means I have literally seen people with their intestines partially extruded from their abdomens who have complained less than you guys are about a $12 test that takes less than 2 hours. After finishing Step 1, Step 2 CK, Step 2 CS, Step 3, your specialty boards (+/- oral exam), possibly your subspecialty boards, and then getting fleeced by MOC for the rest of your careers, CASPer will seem like a youthful opium dream.

Lol really? This is your argument? If you want to go that route, I can tell you I didn’t complain very much when a missile blew up over my head and almost collapsed an entire superstructure on my head. So I guess all of your complaints that you have about medicine are invalid since you’ve never had a missile blow up over your head.

 

[DrStephenStrange]

It means I have literally seen people with their intestines partially extruded from their abdomens who have complained less than you guys are about a $12 test that takes less than 2 hours. After finishing Step 1, Step 2 CK, Step 2 CS, Step 3, your specialty boards (+/- oral exam), possibly your subspecialty boards, and then getting fleeced by MOC for the rest of your careers, CASPer will seem like a youthful opium dream.

I'm from Haiti, I didn't complain growing up poor, and I didn't complain in January 2010 when in 4.3 seconds the world collapsed around me and 300,000+ people died almost instantly which included friends, family, neighbors, and lived in the streets under tents for almost a year, but I sure will complain about this $12 dollar test they are forcing down our throat.

 

[Med Ed]

Lol really? This is your argument? If you want to go that route, I can tell you I didn’t complain very much when a missile blew up over my head and almost collapsed an entire superstructure on my head. So I guess all of your complaints that you have about medicine are invalid since you’ve never had a missile blow up over your head.

My "argument" is simply that it helps to keep things in perspective. I'm sure we can agree on that, at least.

 

[Med Ed]

I'm from Haiti, I didn't complain growing up poor, and I didn't complain in January 2010 when in 4.3 seconds the world collapsed around me and 300,000+ people died almost instantly which included friends, family, neighbors, and lived in the streets under tents for almost a year, but I sure will complain about this $12 dollar test they are forcing down our throat.

Good. Hope it works out for you.

 

[Lawpy]

My "argument" is simply that it helps to keep things in perspective. I'm sure we can agree on that, at least.

That sounds like an excuse for admissions and PDs to add more hoops to jump through and outsource more things to 3rd parties

 

[Matthew9Thirtyfive]

My "argument" is simply that it helps to keep things in perspective. I'm sure we can agree on that, at least.

It does, but trying to shame people into keeping quiet about something they think is a legitimate concern isn’t a legitimate argument.

 

[sunshinefl]

It means I have literally seen people with their intestines partially extruded from their abdomens who have complained less than you guys are about a $12 test that takes less than 2 hours. After finishing Step 1, Step 2 CK, Step 2 CS, Step 3, your specialty boards (+/- oral exam), possibly your subspecialty boards, and then getting fleeced by MOC for the rest of your careers, CASPer will seem like a youthful opium dream.


If you are referring to the paper linked by @WheezyBaby above, that study doesn't even involve CASPer, but rather a homebrew SJT.

I also would not be so quick to judge a test based solely on the r. I'll give you an example. At my institution the MCAT is a poor predictor of preclinical performance as long as the scores are above a certain threshold. Go below that threshold land the MCAT quickly becomes a much stronger predictor of academic difficulties. We see a similar phenomenon with preclinical performance and USMLE scores, with the former being most predictive at the extremes, and much weaker in the middle (which comprises most of the students). Even if these associations are not super strong they are still useful, as they help flag individuals who are at higher risk of needing intervention.

You also have to look at the incremental validity of the test relative to its cost (both monetary and time). A quick, inexpensive test does not necessarily need to add a huge amount of value to a multivariate predictive model in order to justify its use.

To that end, it would be helpful to know what question(s) the people behind the GME pilot are actually asking, and what they are hoping to gain from trying CASPer. But neither of us are privy to that, so we are missing important information.

Look, there are plenty of problems to be had. MMIs all have systematic biases, unstructured interviews don't correlate with anything, but few question them because they are a familiar and time-honored way of doing things. Step scores have been misused for years. At the root of all this evil, however, is the phenomenon of over-application. Since no one wants to try capping residency applications, I think it will take the development of widespread preference signaling to bring this whole mess somewhat under control, and mitigate the need for more GME pilots.

My complaint isn’t the $12 and 2 hours, it’s that they are using an instrument to judge me on some skills, and the exam doesn’t represent my ability of those skills. I took the stupid GME Casper pilot and gave them the same feedback at the end when they asked. It’s not representative.

another complaint, your score is secret from you. Imagine having a scored Step 1 you are applying with, but only the programs can see it. You’re applying blind.

 

[Med Ed]

My complaint isn’t the $12 and 2 hours, it’s that they are using an instrument to judge me on some skills, and the exam doesn’t represent my ability of those skills. I took the stupid GME Casper pilot and gave them the same feedback at the end when they asked. It’s not representative.

another complaint, your score is secret from you. Imagine having a scored Step 1 you are applying with, but only the programs can see it. You’re applying blind.

Please accept my deepest sympathies.

 

[Med Ed]

That sounds like an excuse for admissions and PDs to add more hoops to jump through and outsource more things to 3rd parties.

It does, but trying to shame people into keeping quiet about something they think is a legitimate concern isn’t a legitimate argument.

So what's your plan for ending this madness, fellas?

 

[slowthai]

So what's your plan for ending this madness, fellas?

Don't require stupid tests like casper. Wow. That was so easy. They should pay me for this

 

[Med Ed]

Don't require stupid tests like casper.

And how are you going to make this dream a reality?

 

[Matthew9Thirtyfive]

And how are you going to make this dream a reality?

I mean it’s not going to happen. Some programs will use it just because it’ll cut down applications while pretending the actual reason is because it’s useful at predicting something (even though it is extremely weakly predictive at its best), and the company that administers it is certainly not going to torpedo their profits by admitting its garbage.

 

[Matthew9Thirtyfive]

Please accept my deepest sympathies.

Passive aggressive apathy isn’t a good look on you.

 

[Med Ed]

Passive aggressive apathy isn’t a good look on you.

This entire thread isn't a good look on anyone.