What You Need to Know About the CASPer Test

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

CaffeinatedSquirrel

Married into Medicine
10+ Year Member
Joined
Jan 20, 2013
Messages
1,945
Reaction score
645
By Altus Assessments

CASPer is an assessment tool that is rapidly gaining popularity across the world. By the end of 2017, over 100,000 people have taken the test, and about 50% of all U.S. medical school applicants and 80% of all Canadian medical school applicants will have had to complete CASPer at some point during their application process. CASPer was first created in 2005 at McMaster University and in 2010, the test was incorporated into the official admissions process at the Michael DeGroote School of Medicine. CASPer, which stands for the Computer Based Assessment for Sampling Personal Characteristics, is predominantly a situational judgement test (SJT) that presents test-takers with a series of realistic, hypothetical scenarios and asks them to type out what they would do if they were to be in that particular situation.

This year, CASPer will begin its implementation across a number of physician assistant programs along with many more medical schools and other health care programs (see here for the most updated list of programs who are using CASPer). With its rising popularity, there has also been a rise in the number of articles that have been written about CASPer—which unfortunately have not always been completely accurate. In this article, we want to give you an overview of CASPer straight from the source directly responsible for the construction and delivery of the test.

What is CASPer measuring?

A large portion of a student’s acceptance decision to a medical program is based on their academic achievements, primarily undergraduate GPA and MCAT scores. This ensures that students have a strong foundation of knowledge and demonstrate superior cognitive abilities. However, this is not the only aspect that matters in becoming a successful physician. In fact, one seminal study published in the New England Journal of Medicine found that 94% of identifiable causes leading to disciplinary decisions against doctors were due to lapses in professionalism, not due to lack of cognitive medical competence. This has resulted in a dire need for medical programs to not only assess the cognitive abilities of their applicants, but also ensure that their students possess the interpersonal and intrapersonal skills necessary to become an effective physician. This is where CASPer comes in, as it is an assessment of non-cognitive skills, such as professionalism, communication, ethics, empathy, and motivation, and serves to complement the cognitive assessments that are already required by the programs.

What is the format of CASPer?

CASPer is composed of 12 stations with three probing questions in each station. Eight stations are video-based scenarios and four stations are word-based scenarios. The video-based scenarios are all SJTs whereas the four word-based scenarios are a blend of SJTs and behavioural descriptor questions, which ask respondents to talk about a specific past experience. The scenarios are set in a more general context: in one scenario you may be placed in a workplace setting with your boss, in another scenario you may be placed in a party with friends, in another scenario you may be placed in your own living room with your parents. The scenarios are typically not specific to the clinical setting, as we want to make sure that we are not giving an advantage to those students who have had clinical experience. You can take a look at a number of sample scenarios here at the takeCASPer website. Make sure to always check that the sample scenario and question set is from the official source, as many of the sample CASPer sections from unofficial sources are not representative of the actual test.

How is CASPer evaluated?

Each station of the CASPer test is graded by a different person. This means that every student is assessed by 12 independent human raters who come from a variety of backgrounds (e.g., physicians, nurses, educators, policy makers) and demographics (e.g., race, gender, income) to reflect the diversity of the patient population. Raters are trained to omit spelling and grammatical errors when evaluating responses and to focus solely on the content of the response. The 12 independent ratings are averaged, and scores are then standardized to represent the relative rankings of each student compared to their peers. The scores are automatically distributed to the selected programs within three weeks of completing the test. Students do not receive their results because it is difficult to interpret a single CASPer score without knowledge of how everyone else performed on the test.

How do programs use CASPer in the admissions process?

There is no single answer to this question as each school incorporates CASPer in different ways to guide their admissions process. Most programs use CASPer alongside other metrics like GPA and MCAT scores as a prescreening tool to decide who to invite to the interview process. Some programs use CASPer in a more impressionistic approach, to help facilitate decisions on candidates who are on the cusp of whether they are admitted to the program. Some programs implement a conservative cut-off score for CASPer to screen out applicants, while other programs take a closer look at candidates who have exceptionally high CASPer scores. For instance, Dalhousie Medical School will only consider applicants who score above -1.5 standard deviations from the mean for interview.

What can you do to prepare for CASPer?

Unlike the traditional SJTs with right-or-wrong answers, the primary goal of CASPer is not to examine what you would do in a given scenario, but why you would take that particular course of action. This makes it difficult to study for CASPer, as there is no obvious approach in answering the questions the “right” way. The general SJT literature and our own internal research has also shown that more complicated and challenging SJTs like CASPer are resistant to the effects of practice and coaching. While it may be difficult to improve your CASPer scores in the short-term, we have listed a number of ways students can prepare in advance to ensure a smooth test-taking experience—familiarize yourself with the format of the test, double check to make sure that you meet all the technical requirements (have a functioning webcam!), and plan your schedule well in advance to ensure that you are able to complete CASPer in a comfortable and quiet location.

Conclusion

The admissions process to medical school can be a long and daunting process for many students, and the addition of another assessment tool can be seen as another burden on what is already a fairly strenuous process. However, programs want to gather the most accurate and holistic view of their applicants to ensure that they are making the right decisions, as their students will be shaping the future quality of our healthcare. As technology has given patients easy access to a wealth of information right at their fingertips, they are no longer satisfied with just a medically competent doctor. Patients now seek doctors who are also good listeners, demonstrate empathy and compassion, and provide more personalized care. Historically, the admissions process has done an excellent job in admitting the brightest students, but they are now trying to do a better job in making sure that the students also possess the personal attributes that patients want to see from their physicians.

If you would like to get the most up-to-date information on everything related to CASPer, make sure to follow our blog at takecasper.com and our Twitter account @take_casper. Feel free to directly tweet or email us ([email protected]) with any questions or comments you have about the test.

About the Author

Altus Assessments aims to simplify the admissions process for academic programs while making it more effective by creating tools (e.g. CASPer) that help admissions professionals identify positive personal characteristics in admissions candidates that lead to long-term success.


Read the whole post here.

Members don't see this ad.
 
Last edited:
So apparently according to Altus, "Students do not receive their results because it is difficult to interpret a single CASPer score without knowledge of how everyone else performed on the test." This makes me really angry and it seems like complete BS. If we are are given a score and Altus states above that "scores are then standardized to represent the relative rankings of each student compared to their peers" then it should be fairly easy to tell us our results. In my opinion, the real reason they do not give us our scores is because they do not want us to know how we did on the test. If we knew exactly HOW we scored, we could improve for future attempts.

Also, did anyone else catch that the results of the studies Altus quoted seem to contradict their claim that you can't study for CASPer.

"Due to these concerns, a lot of resources have been devoted to investigating the effects of coaching on most high-stakes admission tests, such as the Scholastic Aptitude Test (SAT), Graduate Record Examination (GRE), Graduate Management Admission Test (GMAT), Medical College Admissions Test (MCAT), and Law School Admission Test (LSAT). An overview of these tests have found that coaching effects on these standardized tests are fairly small (0.25 standard deviation). Additionally, test preparation and retaking does not seem to affect the predictive validity of standardized tests.

When focusing on SJTs in particular, the general literature suggest that coaching effects remain fairly small, with an increase in only 0.5 standard deviation after coaching and an increase in 0.32 standard deviation for students who retook the test in the following admission cycle."

So in other words, the practice effects of coaching are EVEN greater for SJTs like CASPer than they are for MCAT, LSAT, GRE ... etc. Thanks for being so transparent Altus Assessments.
 
Much like applicants should be weary when test prep companies tout significant gains through their test prep products, applicants should also be just as weary when the test creator itself comes out and touts that their test is immune to test prep and that you will benefit little to none from advance preparation.

One study “examined the fakability of an SJT of college students’ performance. Two hundred ninety-three psychology students completed a cognitive test, a personality measure, and an SJT. Only for the SJT, the students were assigned to either an honest or a fake condition. The scores of students in the fake condition were significantly higher than those of students in the honest condition (d = .89). Furthermore, faking had a negative effect on the criterion-related validity (there was a significant drop from r = .33 to r = .09) and the incremental validity of the SJT over cognitive ability and personality”. These results confirm the old adage, “practice makes perfect” and SJTs are part of this norm. It also highlights that SJTs are subject to faking which decreased the criterion-related validity of the test itself.

The reality is that the majority of the research around CASPer has been completed by the creators of CASPer. This is the same group that is currently pushing schools to adopt this test and in the process, spread applicants' wallets even thinner. Their findings on CASPer have not been validated by others.

The creators of CASPer suggest that “if a practice effect exists, then we should see that students do better and better with each section as they get more exposure to the content of the test.” They report they “see absolutely no change in CASPer® scores as students progress through the test” and conclude that there seems to be no evidence of any practice effects with CASPer®. The research shows that practice effects take time to develop and one is not likely to see any practice effects on the same sitting of the same test. However, practice effects for SJTs are well documented. Based on the best research available, the coaching effects on SJTs like CASPer are estimated to be approximately 0.50 SD which is larger than 0.25 SD for other more famous standardized tests such as SAT, GRE, MCAT, GMAT and LSAT.

By the author’s own admission, this suggests that applicants will stand to benefit MORE from coaching on SJTs like CASPer then even from tests such as the MCAT.

Always be critical and do your own research. Good luck!
 
Members don't see this ad :)
To me this test is ridiculous. We have years of GPA, the MCAT, research, volunteering, leadership, shadowing, primary and secondary applications, interviews, etc and now we have to do this CASPER test? They should know by now if we are a good candidate for their school or not.
 
To me this test is ridiculous. We have years of GPA, the MCAT, research, volunteering, leadership, shadowing, primary and secondary applications, interviews, etc and now we have to do this CASPER test? They should know by now if we are a good candidate for their school or not.

I don’t like the test for various reasons, but I agree there needs to be a way to screen out the narcissists and psychopaths, because gpa and MCAT don’t do it. I think mmi is a better tool though.
 
I don’t like the test for various reasons, but I agree there needs to be a way to screen out the narcissists and psychopaths, because gpa and MCAT don’t do it. I think mmi is a better tool though.
It also highly limits people like me who arent the fastest typers. I wish they had a voice recorder so you could answer the questions verbally. I dont know if CASPer would screen out the types of people you mentioned because they would usually manipulate their answers anyways to "fit". I agree that a MMI. All in all, I hate CASPer with a passion mostly cuz we dont know how we do. I would def save myself the primary/secondary money if I bombed CASPer. Its like when people decide to not apply to a school when they get their MCAT score back. But nooooope. And schools cant legally tell you if you did well on it.
 
It also highly limits people like me who arent the fastest typers. I wish they had a voice recorder so you could answer the questions verbally. I dont know if CASPer would screen out the types of people you mentioned because they would usually manipulate their answers anyways to "fit". I agree that a MMI. All in all, I hate CASPer with a passion mostly cuz we dont know how we do. I would def save myself the primary/secondary money if I bombed CASPer. Its like when people decide to not apply to a school when they get their MCAT score back. But nooooope. And schools cant legally tell you if you did well on it.

Yeah, I agree. Let people know how they did. It’s a BS answer saying that the score can’t be interpreted. If you’re afraid of the learning effect, just limit attempts to a single attempt. If it really judges candidates on what it says it does, more than one attempt shouldn’t be necessary.
 
I agree that CASPer should be used a way to screen out psychopaths and those who do really poorly. But other than that I am very skeptical of it as a test that can validly distinguish between those who did alright, well, or very well. Counter to what some others say, coaching is absolutely pointless due to lack of an assigned score or a grading rubric.
 
I have no problem taking a test like that. I just don’t like how the CASPer people do it. I still forked over $30 though.

Doesn't change the fact that it's still another hurdle in this already long process. On the bright side, AAMC will likely be a lot more transparent with applicants.
 
I agree that CASPer should be used a way to screen out psychopaths and those who do really poorly. But other than that I am very skeptical of it as a test that can validly distinguish between those who did alright, well, or very well. Counter to what some others say, coaching is absolutely pointless due to lack of an assigned score or a grading rubric.

CASPer is hand scored, therefore your comment about a lack of grading rubic is not true. Applicants are assigned a score and ranked. If your answers are inferior to others, you rank lower.

Re: coaching being pointless, people once said the same thing about the MCAT in the 1920s when the test was first administered.
 
Yeah, I agree. Let people know how they did. It’s a BS answer saying that the score can’t be interpreted. If you’re afraid of the learning effect, just limit attempts to a single attempt. If it really judges candidates on what it says it does, more than one attempt shouldn’t be necessary.

The fact that applicants are not told how they perform on the test, is one of the strongest indicators that there is a significant practice/coaching effect associated with SJTs....
 
Top