Don't sweat the small stuff (read: CASPer/SJT)

[seanm028]

I got my CASPer results ~2.5 weeks ago and was happy to have received 4th quartile (75–100%ile). Then I got my SJT results today and was a little surprised to see 55%ile. In theory, both tests are supposed to be looking at the same (or very similar) qualities. I took the two tests five days apart, and I don't think my moral/ethical judgment changed considerably during that interval. While this is anecdotal, the take-home point for me is that these tests are very flawed. While the medical school admission process is also flawed, I have enough faith in the system to believe that most schools aren't putting too much stock into either score right now. I recently saw a post from someone who said they participated in their school's admissions process, and the extent of their CASPer usage was something like a 5%ile screen. In other words, as long as an applicant scored above the 5%ile, they passed the screen, and the CASPer wasn't really looked at anymore. If these tests were academic papers, the scientific community would decry their lack of validity and reproducibility, and they would be ignored entirely.

Now, in 5–10 years, medical schools will have had a chance to compare applicants' CASPer/SJT scores to their performance in medical school. If there is a correlation, then we might start to see more emphasis placed on these judgment tests. Of course, by that time, some enterprising individuals probably will have started to "crack the code" to how they are graded and will begin producing prep materials. Until such time, though, I don't think anyone needs to worry too much about their performance on these tests. I see a lot of threads where people ask how heavily schools weigh these tests. The most accurate answer is that nobody knows except the admissions committees, but I suspect the answer is "not very heavily, if at all." The fact that many schools have them as optional goes to show that they're clearly not that big of a deal. Instead, worry about your GPA and your MCAT. Or, you know, your mental wellbeing.

---

Members don't see this ad.

 

[KnightDoc]

Now, in 5–10 years, medical schools will have had a chance to compare applicants' CASPer/SJT scores to their performance in medical school. If there is a correlation, then we might start to see more emphasis placed on these judgment tests. Of course, by that time, some enterprising individuals probably will have started to "crack the code" to how they are graded and will begin producing prep materials. Until such time, though, I don't think anyone needs to worry too much about their performance on these tests. I see a lot of threads where people ask how heavily schools weigh these tests. The most accurate answer is that nobody knows except the admissions committees, but I suspect the answer is "not very heavily, if at all." The fact that many schools have them as optional goes to show that they're clearly not that big of a deal. Instead, worry about your GPA and your MCAT. Or, you know, your mental wellbeing.

I agree that whether and how CASPer is used is a big black box, but where are you getting that ANY, let alone many, schools have it as optional? To my knowledge, all schools either require it or they don't.

SJT is a different story, since it seems to be an AAMC pilot that they are slowly rolling out. It's true that 4 of the 6 SJT schools (hardly many), strongly recommend it as opposed to requiring it, and that certainly implies that test isn't being used to make admission decisions, but it's misleading to lump CASPer in with that just because they are both situation judgment tests.

I don't think a single CASPer school will consider an application complete without it. If they are requiring it, while it is possible they are merely studying it versus using it, you can't possibly know that unless someone somewhere stated that. Am I missing something?

 

[seanm028]

I agree that whether and how CASPer is used is a big black box, but where are you getting that ANY, let alone many, schools have it as optional? To my knowledge, all schools either require it or they don't.

SJT is a different story, since it seems to be an AAMC pilot that they are slowly rolling out. It's true that 4 of the 6 SJT schools (hardly many), strongly recommend it as opposed to requiring it, and that certainly implies that test isn't being used to make admission decisions, but it's misleading to lump CASPer in with that just because they are both situation judgment tests.

I don't think a single CASPer school will consider an application complete without it. If they are requiring it, while it is possible they are merely studying it versus using it, you can't possibly know that unless someone somewhere stated that. Am I missing something?

Fair points. I was lumping the two together for the sake of argument. And on that note, every time I see someone applying to an SJT-optional school ask if they should take the SJT, my answer is a resounding yes. I wasn't intending to discourage anyone from taking it. On the contrary, I would advise anyone applying this cycle to take both tests just to be safe, even if you don't think any of your schools will be looking for it.

I intended to highlight the fact that these tests are probably not outsize factors in any interview or admissions decisions. They may or may not play some role — as you said, it's a black box — but the odds are that they play a small role in comparison to things that the applicant can much more readily control, or at least know (e.g., GPA, MCAT, community service hours, research hours).

Bottom line, I'm advocating for a "balanced" approach. It seems that a lot of posters on here are worried that their 1st quartile CASPer score will sink their application. It probably won't. It seems that you're arguing that we shouldn't swing the pendulum to the other extreme, a point with which I agree.

 

[Goro]

There was an excellent Post in this forum not too long ago demonstrating how how these tests lack any sort of validity as predictors to either being a good doctor, or a good medical student.

In fact, the only data supporting these tests were written by the people who created the tests and hence are marketing them. Kind of a little conflict of interest there, don't you think?

 

[KnightDoc]

Fair points. I was lumping the two together for the sake of argument. And on that note, every time I see someone applying to an SJT-optional school ask if they should take the SJT, my answer is a resounding yes. I wasn't intending to discourage anyone from taking it. On the contrary, I would advise anyone applying this cycle to take both tests just to be safe, even if you don't think any of your schools will be looking for it.

I intended to highlight the fact that these tests are probably not outsize factors in any interview or admissions decisions. They may or may not play some role — as you said, it's a black box — but the odds are that they play a small role in comparison to things that the applicant can much more readily control, or at least know (e.g., GPA, MCAT, community service hours, research hours).

Bottom line, I'm advocating for a "balanced" approach. It seems that a lot of posters on here are worried that their 4th quartile CASPer score will sink their application. It probably won't. It seems that you're arguing that we shouldn't swing the pendulum to the other extreme, a point with which I agree.

Bingo! Unlike GPA and MCAT, it never occurred to me to wait for my CASPer score before submitting, since they never even released the quartile to everyone before this year.

I agree with everything else you said. In Canada they swear by it, but schools in the US are still trying to figure it out. There is no transparency, so we have no way to know whether or how it is being used. Some schools are undoubtedly using it to screen people out, while others equally clearly are not. It also seems to be crap, but some schools are apparently attracted by its promise to be an objective, non-cognitive metric that increases diversity.

 

[seanm028]

At the end of the day, I respect what is trying to be accomplished with these tests. Most everyone agrees that this process needs to be more humanistic, and, at least in theory, CASPer is trying to accomplish just that. I'm not sure if the mark is being hit right now (due to the lack of transparency), and it has definitely been a frustrating experience as a test-taker. But for the sake of improving the admissions process, I really do hope that they figure it out.

 

[seanm028]

There was an excellent Post in this forum not too long ago demonstrating how how these tests lack any sort of validity as predictors to either being a good doctor, or a good medical student.

In fact, the only data supporting these tests were written by the people who created the tests and hence are marketing them. Kind of a little conflict of interest there, don't you think?

Mmhmm. I like to think that the medical schools right now are more concerned with just collecting data points so that in 5–10 years, they can make statistically sound decisions about how these scores may or may not correlate with any reasonable measures of success.

My point in starting this thread was to provide a single example of the lack of reproducibility of these tests, hoping that that would help some people lose less sleep over their scores. I would hope that any reasonably intelligent admissions officer would put very little weight on the scores for the next several years, given their obvious lack of validity.

 

[deleted1025488]

At the end of the day CASPER seems to be another one of many factors that we premed can't control. I see it best we collectively just ignore this piece of trash and leave it where it belongs....

 

[effectedtag]

In the UMiami thread someone asked about CASPER and this was their response: "Thank you for your interest in the University of Miami Miller School of Medicine. A low CASPer score may not exclude you from an interview, but with 11,000 applicants it will not help."