New admissions metric: your ideas

[wiloghby]

---

Members do not see this ad.

In the AAMC's "AAMC Admissions Initiative: A Pathway to Competency-Based Admissions", (https://www.aamc.org/download/308462/data/admissionsinitiativesummary.pdf), the AAMC outlines competencies correlated with success in medical school and in the "healthcare system of the future":

Interpersonal Competencies
Service orientation
Social and interpersonal skills
Cultural competence
Team work
Oral communication

Intrapersonal Competencies
Integrity and Ethics
Reliability and dependability
Resilience and adaptability
Capacity for improvement

In short, they want to modify the application process to give each applicant a quantiative measure of the above qualities alongside the traditional GPA and MCAT numbers. To do that, AAMC is trying to find a way to test or measure these competencies quantitatively, so the scores can be looked at by AdComms prior to sending out interview invites:

"As highlighted above, medical schools face a significant challenge in trying to efficiently learn more about applicants during the screening process that precedes inviting applicants to interview. Our goal, over the next few years, is to investigate, test and deploy a measurement instrument (or instruments) that will provide medical schools with better and timely information about the intrapersonal and interpersonal competencies identified above."

My question to anyone who cares: what would do you think the ideal "measurement instrument" for the above qualities/competencies should look like in the application process? What do you think would work the best?

(It reads to me like they are leaning towards a personality test of some kind, or a survey of those who know you?)

 

[ærospallo]

This was published in 2012?

https://www.aamc.org/download/308462/data/admissionsinitiativesummary.pdf

 

[Boolean]

I think they may incorporate some ethics questions. I don't think they would do a personality test or survey. Perhaps an interview? MMI?

 

[repititionition]

...prior to sending out interview invites...

there is a tool that doesn't meet the bolded criteria but does meet most of the others mentioned: MMI

Prediction: within 10 years >50% of schools will be incorporating MMI - perhaps not exclusively, but incorporating it - in their interview process.

 

[LizzyM]

I do think that an interview can capture some of these attributes, including oral communication and teamwork (where the interview involves a group activity). Some well targeted questions can assess some of the others.

 

[wiloghby]

there is a tool that doesn't meet the bolded criteria but does meet most of the others mentioned: MMI

Prediction: within 10 years >50% of schools will be incorporating MMI - perhaps not exclusively, but incorporating it - in their interview process.

This is precisely the reason I bolded that part, because indeed I do think you are right... the interview can measure some of these things, as LizzyM said. But it sounds like the AAMC wants to get some idea of that *before* an AdComm wastes valuable interview resources (and the applicant wastes hundreds of dollars getting there) on an applicant who is a poor fit from the get go.

 

[CyberMaxx]

I interviewed at a school which was entertaining the idea of using a personality test taken on a computer as a component of the interview. My feeling was that a personality test may be a little too far.

 

[wiloghby]

I interviewed at a school which was entertaining the idea of using a personality test taken on a computer as a component of the interview. My feeling was that a personality test may be a little too far.

I agree, and I've seen a couple links to papers trying to correlate different personality qualities, as measured by test xyz, with healthcare success, and the results make me think it's a sound thing to do based on the data...

..but in practice, it seems very creepy.

In a way, I hope things aren't going in that direction...but I also think targeting those competencies would mean we'd have some very amenable classmates to work with. I hope there's a less creepy solution that would work for a pre-interview metric.

 

[repititionition]

Interpersonal Competencies
Service orientation
Social and interpersonal skills
Cultural competence
Team work
Oral communication

Intrapersonal Competencies
Integrity and Ethics
Reliability and dependability
Resilience and adaptability
Capacity for improvement

...or a survey of those who know you?...

It says a lot about the current system that these are not being effectively captured in LOR already.

The rubricization (spun that from whole cloth) of the "average" LOR has stripped it of most of its value. That is to say, a very few capture the golden triad:

1) intimate - the writer a) knows the recipient well b) for a sufficiently long time and c) in a context relevant to the candidate's suitability for medicine
2) explicit - the writer effectively articulates the distinguishing positive characteristics of the applicant
3) appropriate - the writer does this in a way that an AdCom can parse

Having now seen a few LOR (beyond my own - and breathe a collective gasp, SDN, because I SAW MOST OF MY OWN LETTERS!), it seems that many rely almost exclusively on point 3; the writer knows the lingua franca of the medical school LOR and essentially plays a game of madlibs, inserting adjectives or vignettes as they're able.

This likely results from a deficit in points 1 (a failure of the student in asking a writer that does not know them well enough, or lack of -anyone- in a position of authority knowing them well enough), and point 2 (with similarly shared responsibility - either the student is just not that exceptional [we can't all be] or the writer is not that passionate or articulate).

The LOR has been reduced, by and large, to the lowest common denominator - something that often only rules out a very marginal minority rather than serving as an actual point of differentiation between otherwise deserving/qualified applicants. A meaningful LOR, on the other hand, would capture most of the attributes desired, with a whole lot less headache and gamesmanship than something like a personality test or an additional layer of quantitative screening on the dimensions listed above.

 

[leggo_my_ecmo]

This sounds a lot like CASPer, which McMaster has been using for a while now. Mac also was the first school to implement PBL and then MMIs, so something akin to CASPer will probably become widespread as well.

 

[Goro]

100% concur with my learned colleague. There are some things you simply can't pick up on in a 1:1 interview.

I do think that an interview can capture some of these attributes, including oral communication and teamwork (where the interview involves a group activity). Some well targeted questions can assess some of the others.

 

[wiloghby]

This sounds a lot like CASPer, which McMaster has been using for a while now. Mac also was the first school to implement PBL and then MMIs, so something akin to CASPer will probably become widespread as well.

Yep.. this is basically what I envisioned when I read the AAMC document. I guess it's already happening. I would be very surprised if it's not widespread (though far from universal) in a decade.

For what it's worth.. I really don't like MMI either. I was just curious what SDNers would come up with.

 

[Narmerguy]

there is a tool that doesn't meet the bolded criteria but does meet most of the others mentioned: MMI

Prediction: within 10 years >50% of schools will be incorporating MMI - perhaps not exclusively, but incorporating it - in their interview process.

Alternate prediction: Within 15 years the AAMC (or a similar organization) will create a standardized MMI service that applicants sign up for, perhaps in conjunction with their MCAT. In addition to the MCAT score, schools will pay a fee to have access to the MMI results.

A questionnaire of sorts seems too easy to "game", but perhaps I'm wrong. It seems like it would lead to an arms race of Kaplan-like courses training students to answer questionnaires in a way that shows leadership, empathy, compassion, altruism, teamwork, etc etc. It's one thing to check a box indicating what one thinks is the "teamwork friendly" answer, and another to actively show that one can work in a team.

 

[Long Way to Go]

Some of the jobs I have applied for in the past asked for a personality test (took like half an hour+, kind of annoyingly long). I knew it was overall hospital policy, not specific to this one job, but it kind of turned me off. It was just too easy to answer what they were looking for, which you can't do in an interview setting without seeming disingenuous.

 

[MadisonMG]

Looks like Yale and U of M too.

 

[PantherPride]

It says a lot about the current system that these are not being effectively captured in LOR already.

The rubricization (spun that from whole cloth) of the "average" LOR has stripped it of most of its value. That is to say, a very few capture the golden triad:

1) intimate - the writer a) knows the recipient well b) for a sufficiently long time and c) in a context relevant to the candidate's suitability for medicine
2) explicit - the writer effectively articulates the distinguishing positive characteristics of the applicant
3) appropriate - the writer does this in a way that an AdCom can parse

Having now seen a few LOR (beyond my own - and breathe a collective gasp, SDN, because I SAW MOST OF MY OWN LETTERS!), it seems that many rely almost exclusively on point 3; the writer knows the lingua franca of the medical school LOR and essentially plays a game of madlibs, inserting adjectives or vignettes as they're able.

This likely results from a deficit in points 1 (a failure of the student in asking a writer that does not know them well enough, or lack of -anyone- in a position of authority knowing them well enough), and point 2 (with similarly shared responsibility - either the student is just not that exceptional [we can't all be] or the writer is not that passionate or articulate).

The LOR has been reduced, by and large, to the lowest common denominator - something that often only rules out a very marginal minority rather than serving as an actual point of differentiation between otherwise deserving/qualified applicants. A meaningful LOR, on the other hand, would capture most of the attributes desired, with a whole lot less headache and gamesmanship than something like a personality test or an additional layer of quantitative screening on the dimensions listed above.

Perhaps if the AAMC had a standardized LOR service these problems could be avoided. Say there was a survey in addition to/replacing the LOR that had letter writers evaluate the applicant in each competency along with text boxes for expanding upon their ratings, to prevent spamming the highest ratings. Each LOR would directly address these characteristics for each applicant and you could avoid all the fluff. You also have a way of estimating these characteristics before the II in a way thats much harder for an applicant to game than, say, a questionnaire or personality test.

 

[Goro]

Concur with my learned colleague. One would be surprised at how you can learn who is a team player and who isn't in a group interview setting.

I do think that an interview can capture some of these attributes, including oral communication and teamwork (where the interview involves a group activity). Some well targeted questions can assess some of the others.

 

[Holmwood]

Maybe instead of a personality test of sorts, you submit a mini description about an experience for each of those 9 qualities. They will then have 3 people who will rate it 1-5 then average it.


Unless they wanted us to pay for an MMI sort of session (like extra 50 bucks) and have people rate our performance.

 

[gettheleadout]

100% concur with my learned colleague. There are some things you simply can't pick up on in a 1:1 interview.

Concur with my learned colleague. One would be surprised at how you can learn who is a team player and who isn't in a group interview setting.

So nice he concurred twice!

 

[Lucca]

I do think that an interview can capture some of these attributes, including oral communication and teamwork (where the interview involves a group activity). Some well targeted questions can assess some of the others.

Actually, I'm curious as to how group interview activities can accurately display teamwork. The team is 1) unfamiliar with one another 2) competing against each other and 3) each applicant is also trying to distinguish themselves as a leader and an individual. It's essentially a very, very bad environment for good teamwork to occur. Wouldn't it be better to just have lots of team-based experience on your resumé?

 

[Goro]

You see who is unwilling to listen to the others, for starters. Body language and eye contact are more apparent. If the entire group is asked a question, and told that each answer has to be different, critical thinking skills are readily discernable. We don't ask interviewees to role play, but some interviewers elsewhere do, and then participatory skills (or lack of) are immediately visible.

The interviewees are also NOT competing with each other! Admissions isn't a zero-sum game...all would be accepted if they're good interviewees

Actually, I'm curious as to how group interview activities can accurately display teamwork. The team is 1) unfamiliar with one another 2) competing against each other and 3) each applicant is also trying to distinguish themselves as a leader and an individual. It's essentially a very, very bad environment for good teamwork to occur. Wouldn't it be better to just have lots of team-based experience on your resumé?

 

[Ad2b]

role play with currently attending medical students would be a means as well... no competition as they are already attending and know full well what "fit" means at their particular school.

 

[Easonred57]

NYMC now requires CASPer with their secondary application... this should be interesting.

https://www.nymc.edu/Academics/SchoolOfMedicine/Admissions/SecondaryApplication.html

 

[ChillDawg]

Alternate prediction: Within 15 years the AAMC (or a similar organization) will create a standardized MMI service that applicants sign up for, perhaps in conjunction with their MCAT. In addition to the MCAT score, schools will pay a fee to have access to the MMI results.

A questionnaire of sorts seems too easy to "game", but perhaps I'm wrong. It seems like it would lead to an arms race of Kaplan-like courses training students to answer questionnaires in a way that shows leadership, empathy, compassion, altruism, teamwork, etc etc. It's one thing to check a box indicating what one thinks is the "teamwork friendly" answer, and another to actively show that one can work in a team.

Ugh, could you imagine the cost?

 

[PreMedicalStudent89]

NYMC now requires CASPer with their secondary application... this should be interesting.

https://www.nymc.edu/Academics/SchoolOfMedicine/Admissions/SecondaryApplication.html

What is CASPer for?? Can someone explain it to me??

 

[jayceee]

What is CASPer for?? Can someone explain it to me??

http://fhs.mcmaster.ca/mdprog/casper.html

McMaster University started it a few years ago. It's a computer based personality (non-cognitive and interpersonal skills) test.

McMaster also pioneered the MMI interview.

But don't worry, a quick Google search revealed that there are already multiple companies/websites out there willing to help anxious pre-meds conquer test.

 

[Affiche]

Trying to demonstrate teamwork in a setting where you're being interviewed cannot possibly be representative of your actual ability to work as part of a team. It's too unnatural.

As @Lucca mentioned, everyone is trying to distinguish himself as a leader, which is inherently not very "team-like". Despite trying to be a leader, no one in their right mind would take a strong stance and challenge other members' ideas out of risk of appearing too argumentative, so the group succumbs to groupthink and goes with whatever mediocre idea comes up first. It's a very poor and unrealistic representation of one's interpersonal and critical thinking skills.

 

[Affiche]

And don't even get me started on role playing. That has to be the poorest form of extrapolating data of all interviewing practices to date.

 

[Rainbow Zebra]

What is CASPer for?? Can someone explain it to me??

See the link, a nice explanation of CASPer. Basically a mini personality test.

 

[Advisor Prep]

What is CASPer for?? Can someone explain it to me??

CASPer is basically an MMI on the computer. We've been helping applicants conquer this beast for years since it was implemented for admissions to McMaster Medical school a few years ago!

It looks like the only two US schools using CASPer right now are NYMC and Rutgers. Good luck!

 

[deleted393595]

And don't even get me started on role playing. Role playing has to be the poorest form of extrapolating data of all interviewing practices to date.

A Dungeons & Dragons based group-interview would really determine how people operate under great adversity!

Either that or Mafia.

 

[Affiche]

A Dungeons & Dragons based group-interview would really determine how people operate under great adversity!

Either that or Mafia.

Maybe it's because I'm a quarter Russian, but Mafia is my ****.

 

[Ad2b]

A Dungeons & Dragons based group-interview would really determine how people operate under great adversity!

Either that or Mafia.

Just play WoW and scroll the /2 ... that'll say a lot about the person. Not that I would know anything about that sort of game ...

 

[mcatjelly]

CASPer is basically an MMI on the computer. We've been helping applicants conquer this beast for years since it was implemented for admissions to McMaster Medical school a few years ago!

It looks like the only two US schools using CASPer right now are NYMC and Rutgers. Good luck!

Unclear how a personality test is "conquerable" unless you're telling premeds how to answer in a certain way to get a certain result, in which case the ethics of doing that are highly questionable.

 

[Sourgrapesss]

Unclear how a personality test is "conquerable" unless you're telling premeds how to answer in a certain way to get a certain result, in which case the ethics of doing that are highly questionable.

People will do that anyways..
Oh just read advisor preps comment. Lol of course there would be $1000 classes for a personality test. I agree that it's unethical.

 

[Advisor Prep]

Unclear how a personality test is "conquerable" unless you're telling premeds how to answer in a certain way to get a certain result, in which case the ethics of doing that are highly questionable.

Hi mcatjelly,

Both CASPer and the MMI were designed with similar goals in mind revolving around professionalism. Our founder has worked with Dr. Harold Reiter, been involved in admissions at McMaster medical school, and taught their medical students. We have unparalleled expertise in these areas of the medical school admissions process.

Regarding your comment on the ethics, professional preparation services for CASPer and the MMI are no different than preparation services for the MCAT, LSAT, GMAT etc. We do occasionally offer pro-bono services for applicants that wish to gain this same competitive edge .

Some may consider CASPer a personality test because it evaluates an applicant’s personal and professional characteristics. However, according to the founders of CASPer, in the medical profession alone, a lack of professionalism is a $50 billion problem – accounting for 92% of complaints. This is truly at the heart of the CASPer test.