Med School Admissions Process = Load of Crap

[roboyce]

Shredder,

Allow me to reframe the argument. Let's take the show "The Apprentice." Yes, the Donald Trump version. He starts out with what we might call an applicant pool of candidates, many of whom come from diverse backgrounds, with a variety of experiences. Some applicants have outstanding credentials from world class ivy-league institutions, while others have limited formal education, but other, less tangible qualities. Now, if the Donald followed your logic, he would simply pick out those with the top scores from the best MBA schools. He could just look over the applicants' paperwork (let's refer to it as AMCAS) and make his final decisions about who to hire and who to fire. Hell, the show would be over in one episode. But do you really think he would have ended up with the best people? Absolutely not.

Of course, you can argue that the Donald isn't completely score oriented, but rather, results oriented, to paraphrase you. So, while he takes into account the the objective success (GPA/MCAT) of the candidates he also wants to find people who will turn a profit when the time comes to lay down his own capital. The candidates demonstrate their potential by showing leadership, initiative, and creativity through a variety of tasks organized by the Donald (or most likely by the producers, but let's not lose focus here). Now I think it becomes quite clear that those with the best scores are not always those who succede in these situations. Rather, it is the intangible qualities that often make or break the candidate. In fact, I might even be so bold as to say that the scores (GPA/MCAT) are probably the least relevant element of each applicant when it comes down to the ultimate success at each task.

Now, let's say that the Donald didn't have time watch the candidates take part in each task. After all, he is a busy man. So instead, he asked a few people to write a summary of how each candidate performed under the conditions, emphasizing those subjective qualities that tend to produce good businessmen (LORs). In addition to that, he also asks each candidate to describe his own involvement in each task, outlining individual responsibilities and performance (AMCAS ECs). While this process would not always prove as effective as personally observing each candidate, given the feasibility of screening more than 4,000 applicants, it is the best substitute.

Does this process look familiar? Over and over, you argue in favor of a results oriented system, but I don't think you are accurately establishing what results the medical community is seeking. While I am not the one to outline the exact terms of that goal, I can tell you that it is NOT to create a pool of doctors who are excellent test takers or academic all-stars, which is essentially what your studies show. Rather, a diverse pool of physicians is needed to meet the variety of challenges facing America. No one set of skills is going to accomplish that task. The Donald recognizes this, as do the adcoms, so why does it confound you?

---

Members don't see this ad.

 

[RxnMan]

I agree with Shredder in some ways. I don't like subjective measures being used; spending a day in a clinic with a nurse is not the same thing as spending two years in an ER, but it can be weighed the same for the clincal experience requirement.

I also like the idea of allowing the school and the profession to weed out those who can't hack it, instead of placing gatekeepers and retaining high numbers of students. My alma mater took tons of money from freshmen who never came back for sophomore year. It didn't mean a thing that you got admitted to my university. If you were one of the <30% of your entering class who graduated, THAT meant something. Except for two problems.

Medical education is expensive. Not just to us, but to the institution. Docs are all busy - anyone who's shadowed one knows this. To pay one to teach a class is expensive. Especially to pay one that mentors a student (a la MS1 and MS2 programs) and then the student leaves. Where has that investment gone? There is no such thing as a 1/2 M.D.. You either are a doctor, or you aren't. The curriculum is so specialized that you can't transfer those credits to another program (though, in these days of increasing care extenders, it might be worth exploring the idea of changing curricula to allow leaving med students transferring into a P.A. or R.N. program*). So instead of paying 400 physicians to mentor 400 MS1s + 300 to mentor 300 MS2s (~100 students leaving/year to get a graduating class of ~100 from 400 starting), schools have gatekeepers, and only employ 200 total to mentor 200 MS1-MS2. The same arguement could be extended to the clinical years. How can you finance all of the physicians to spend time taking on all of those students, especially since you know that a significant number will leave? Clinical rotations are also physically limited. There's just not enough teaching hospitals to teach everyone under an "selection by attrition" educational model. By dedicating a few physician's time to create a gatekeeper ADCOM, the school may actually be saving money.

The second issue is ethical. Teaching hospitals bear a burden by teaching inept students how to become competent physicians. I don't think it's ethical (or moral) to increase the patient's exposure to risk by increasing the number of students. Especially by creating a glut of inexperienced kids that have to be watched by an already strained medical staff. Conversely, it wouldn't be good to create a stream of half-trained docs that could conciveably provide poor care to the public.

On interviews, think about this: everyone wants to be a doc. It is a buyer's market (since applicants "sell themselves" to the committees). Selection must be done by some method. Grades and MCAT are not enough to reduce the number of qualified applicants, so some other metric must be used. They add ECs, research experience, etc.. Considering that docs will be cutting into people, an interview with someone who, up until that point, only exists on paper, does not seem to be that bad of idea.

*This was not meant to insult any of the wonderful and dedicated people in these challenging programs

 

[Law2Doc]

http://www.springerlink.com/(3q1y5i...nal,7,32;linkingpublicationresults,1:102840,1

Abstract
Purpose: Determining the valid andfair use of the interview for medical schooladmissions is contingent upon a demonstrationof the reproducibility of interview scores. This study seeks to establish thegeneralizability of interview scores, firstassessing the existing research evidence, andthen analyzing data from a non-experimentalindependent replications research design.

Methods: Multivariate andunivariate generalizability analyses areconducted using data from a structuredinterview obtained from a population of medicalschool applicants over two years.

Results: The existing literaturedoes not provide sufficient evidence regardinginterview reliability. In this study,interview scores derived from a standardizedinterview were found to display low to moderatelevels of reliability. Interview scores do notappear to possess the level of precision foundwith other measures commonly used to facilitateadmissions decisions.

Discussion/Conclusion: Given theresults obtained, the fairness of using theinterview as a highly influential component ofthe admission process is called into question. Methods for using interview data in apsychometrically defensible fashion arediscussed. Specifically, attention to decisionreliability provides guidance on how interviewscores can best be integrated into theadmissions process.

For what it's worth, this study was already brought into question back in posts 80 and 81 based on that fact that now, almost two years later, these SAME AUTHORS seem to have apparently embraced the interview as a tool, and have just published an article (abstract cited in those posts) suggesting ways the interview might be improved. (I like the above posters analogy about the apprentice as well.)

 

[Shredder]

that abstract was inferior to the other paper i posted, replete with the table of correlations showing the limited usefulness of the interview compared to other measures. this isnt the apprentice, where you have to select a single candidate for a single high level position. its med school, where you pick over a hundred people for lowly entry level positions at schools. and, in the apprentice candidates' objective accomplishments are very meaningful. the winner of the first one had previously started a multimillion dollar business. business winners have strings of positive numbers that they leave in their trail. once again its about more than how they smile or shake hands and charm people. also, the apprentice is television--the candidates on the show are not even the best applicants. they are the mix of people who make the best entertainment, just like the Real World on mtv. embracing this and that measure in papers is one thing, but producing numerical data that validates embraces is more difficult. studies always show that subjective measures are bogus and beaten by objective ones.

rxnman i knew the financial arguments for gatekeeping would come up sooner or later. but, doesnt that imply that med schools are losing money? meaning, the students are reaping more benefits than they are paying costs (during the course of med school)? this might be understandable for public schools that are subsidized by the state, but for private schools it just doesnt seem to make sense.

the MCAT (or some other super test) is enough to select applicants. thats why the its graded on a curve. to separate wheat from chaff. donald might quite possibly narrow down applicants largely based on a test, but once again the govt bars that. fact: jews have the highest average IQ of all peoples, and they are grossly overrepresented in prominent positions as a putative result, including in big business and among billionaires. coincidence? all of these objective measures boil down to cognitive abilities, which are ultimately the best predictors of success in anything (even quarterbacking to some extent) except when you are dealing with very isolated and rare cases like--i dont know, richard branson maybe. and you cannot base admissions or rules on the premise of selecting those isolated and rare cases, bc nobody has the talent to pick those people. standardized testing assumes that you are dealing with a large sample size and trying to place a large number of ppl on a gradient--which is exactly what med school or any admissions process is. i dont see people in other countries dying and showing poor health stats bc they have bad docs that were not selected properly--on the contrary their stats are superior and with lower costs, and inferior docs do not generally create those results

 

[Law2Doc]

that abstract was inferior to the other paper i posted, replete with the table of correlations showing the limited usefulness of the interview compared to other measures. this isnt the apprentice, where you have to select a single candidate for a single high level position. its med school, where you pick over a hundred people for lowly entry level positions at schools. and, in the apprentice candidates' objective accomplishments are very meaningful. the winner of the first one had previously started a multimillion dollar business. business winners have strings of positive numbers that they leave in their trail. once again its about more than how they smile or shake hands and charm people. also, the apprentice is television--the candidates on the show are not even the best applicants. they are the mix of people who make the best entertainment, just like the Real World on mtv. embracing this and that measure in papers is one thing, but producing numerical data that validates embraces is more difficult. studies always show that subjective measures are bogus and beaten by objective ones.

rxnman i knew the financial arguments for gatekeeping would come up sooner or later. but, doesnt that imply that med schools are losing money? meaning, the students are reaping more benefits than they are paying costs (during the course of med school)? this might be understandable for public schools that are subsidized by the state, but for private schools it just doesnt seem to make sense.

the MCAT (or some other super test) is enough to select applicants. thats why the its graded on a curve. to separate wheat from chaff. donald might quite possibly narrow down applicants largely based on a test, but once again the govt bars that. fact: jews have the highest average IQ of all peoples, and they are grossly overrepresented in prominent positions as a putative result, including in big business and among billionaires. coincidence? all of these objective measures boil down to cognitive abilities, which are ultimately the best predictors of success in anything (even quarterbacking to some extent) except when you are dealing with very isolated and rare cases like--i dont know, richard branson maybe. and you cannot base admissions or rules on the premise of selecting those isolated and rare cases, bc nobody has the talent to pick those people. standardized testing assumes that you are dealing with a large sample size and trying to place a large number of ppl on a gradient--which is exactly what med school or any admissions process is. i dont see people in other countries dying and showing poor health stats bc they have bad docs that were not selected properly--on the contrary their stats are superior and with lower costs, and inferior docs do not generally create those results

Um - maybe I'm dense, but I don't see how Canada's licensing exam is any sort of measure of the overall quality of US physicians. I think a lot of us would agree that there is likely some correlation between MCATs and those who do well on boards. Smart folks and good test takers would tend to do better than average on both. But again, you hit the snag that high scorers and good test takers do not necessarilly equal good doctors. You are running circles if you say that med school should accept folks who do good on tests so that they can subsequently do well on the next set of tests -- that is clearly not the ultimate goal.

 

[RxnMan]

rxnman i knew the financial arguments for gatekeeping would come up sooner or later. but, doesnt that imply that med schools are losing money?

Schools generally don't lose money, but that doesn't mean that they turn a profit, either. To my knowledge, all medical schools are all not-for-profit institutions, both public and private. Cost-cutting measures in one department (gatekeeping) allows for expansion in others (getting that Nobel laureate professor, or a new cancer wing). The difference between public and private tuition is the lack of state support, which the institution generally passes on to the student, lowering their bill.

The above arguements don't even touch on my other issues: teaching space is physically limited, increasing the number of students beyond what can be monitored by physicians exposes the patients to high risks, and that the value invested in a medical student is lost if they drop out.

 

[happydays]

Shredder, the interviews in the business world is completely subjective, too. A qualified candidate can't lack anything in terms of brains and personality. Just like the med admissions process: your scores get you the interview, and the interview determines who's got the personality.

The verbal on the MCAT only determined how well someone can read and answer questions, and the written portion is just a joke (it makes sure that the candidate can write with English). Both of those can't be good measurements of someone's ability to TALK to patients, to convey trust.

In the business world, a top candidate is smart AND can lead a diverse team of PEOPLE, and here's where the communications skills come in, which is tested during the interview.

 

[LizzyM]

It would be easy to expand the size of the lecture halls and the anatomy labs to accomodate more students with the plan to flunk out a large proportion of them before the Boards. The self-limiting step in med admissions is the number of "slots" for M3 and M4 students in clinical settings. You can't very well have an entourage of 30 students trailing a team and expect anyone to learn to do anything.

The real crime in that approach would be the heavy burden of student loans taken to pay for the first 2 years of medical school without the earning power of the "M.D." to ease the repayment of the loans.

According to the AAMC, in 2005, 45% of all applicants matriculated. The mean (I'd rather median but I have to take what I can get) MCAT of those who applied was V: 8.9, P: 9.1, B: 9.5. So, if we had Shredder's system, only the applicants with the highest MCATs would get in and the rest would walk. If we are taking the top 45% that would correspond, roughly, to the mean MCAT of all applicants. This would correspond to everyone with V:9, P9, B, 10 or greater getting. So, if you had less than those numbers & got in: consider yourself lucky, you got an advantage because things other than MCAT are used to select a class. If you had those numbers or better and got at least one offer then the system works as you'd like it to -- quit your b*tchin'. If you had numbers in that range or better & didn't get in, you didn't apply to a range of schools (you won't get into the top 10 schools with those "average" numbers), or the other factors in admission decision (gpa, LOR, PS, EC, interview) worked against you.

Am I missing something?

 

[roboyce]

this isnt the apprentice, where you have to select a single candidate for a single high level position. its med school, where you pick over a hundred people for lowly entry level positions at schools. and, in the apprentice candidates' objective accomplishments are very meaningful. the winner of the first one had previously started a multimillion dollar business.

So are you saying there are no medical school applicants out there with accomplishments that are very meaningful? Sure, I'll give you that following a doctor around for x number of hours isn't particularly impressive, neither is generic undergradute research, but you have to recognize that there are quite a few candidates who have undertaken significant activities such as starting a business. The adcoms recognize and value this experience, as they rightly should. Now, if you simply want to compare traditional student A to traditional student B, both with x hours of shadowing, y hours of volunteering, and z number of publications, then your argument may gain some strength, but the fact is that the great majority of applicants have SOME unique aspect of their application.

the MCAT (or some other super test) is enough to select applicants. thats why the its graded on a curve. to separate wheat from chaff...all of these objective measures boil down to cognitive abilities, which are ultimately the best predictors of success in anything (even quarterbacking to some extent)

So...I scored a 36 on the MCAT. You scored a 37. Does that mean that you are wheat and I am chaff? In fact, I seriously question your assertions here. Why don't you click on my mdapps link and then we'll discuss who is ultimately more prepared to find success in leadership, medicine, or business (yes, that comment was extremely arrogant, but I only used it to illustrate a point - I apologize to everyone). You're forgetting the very essence of the "Bell Curve" philosophy that you are attempting to espouse...that the statistical predictions say absolutely NOTHING about the future course of any INDIVIDUAL.

 

[Gut Shot]

Too bad I've earned a coveted spot on Shredder's ignore list. This article might have helped him sleep better at night.

The role of grades in gaining admission to highly selective medical schools.
Cooke WD, Fontenella D, Cooke WD. Acad Med, 67:846-9, 1992.

The authors examine the role of grades in the admission decisions at a group of 19 highly selective medical schools by analyzing over 8,000 applications from Cornell University students for the entering medical school classes of 1982 through 1989. The results illustrate the great influence of the grade-point average (GPA) on the admission decision. Between the GPA levels of 3.0 and 3.8, the chance of acceptance increased by a factor of about two for each increment of .2 in the GPA. For a subset of the nine most selective of the 19 institutions, the chance of acceptance increased by a factor of five for each increment of .2. At these nine schools, of 1,157 applications with a GPA of less than 3.4, only four were approved. The authors suggest the evidence indicates that students often receive encouragement to continue the application process even though the chances of eventual acceptance are negligible.

 

[Shredder]

So...I scored a 36 on the MCAT. You scored a 37. Does that mean that you are wheat and I am chaff? In fact, I seriously question your assertions here. Why don't you click on my mdapps link and then we'll discuss who is ultimately more prepared to find success in leadership, medicine, or business (yes, that comment was extremely arrogant, but I only used it to illustrate a point - I apologize to everyone). You're forgetting the very essence of the "Bell Curve" philosophy that you are attempting to espouse...that the statistical predictions say absolutely NOTHING about the future course of any INDIVIDUAL.

gahhh ive gotten myself in too deep. you have to put in a fudge factor for any test scores, like + or - such and such amount. so no 1 point differential shouldnt be grounds for rejecting or accepting this and that person. no youre not wheat and im not chaff, or whatever, you know what i mean. i didnt want to turn this talk personal, thats when things go downhill. but personally i do practice what i preach and never have qualms with ppl getting in who have higher mcat scores than me. bell curve yes youre right.

look everybodys pushing me into this corner, but my point is that admissions have grown too subjective and need to be more systematic and transparent. i would like to see some studies and data bc rhetoric is not leading this discussion anywhere but back and forth or pushing shredder into a corner--and no its not bc im necessarily wrong; its bc human nature, whether rational or not, is often to side with 10 peoples rhetoric vs 1 persons data. clearly no one will be persuaded by words alone, neither me nor any of you all, and in that case you must turn to the models and studies to determine things empirically

look roboyce starting a business is cool, im sure youre legit, lets not make beef in this talk. bell curve was a good book and good science, it was very systematically done and transparent unlike what med school admissions seem to be--shrouded in secrecy and the random factor. that may be ok for grad school of astrology but i expect more from med schools who espouse the scientific method, research and whatnot

rxnman yes youre right that there are logistical concerns about bumping up class size, but i mentioned earlier that it was a minor point in the thread and probably deserving of its own discussion. mostly food for thought and bringing up the specificity vs sensitivity issue--which also applies to pharmaceuticals and regulation if you know what i mean

so ill reiterate--lets talk about this in a detached manner (if i fail to do so then call me out on it) and cite some numbers rather than only words. its my suspicion that you guys will have a hard time finding numbers to back up your assertions, and sorry but research can never be conducted that way. look, we dont evaluate businesses based on how flashy their image or logo is, how nice their employees are, or how tall their CEO is. in the end all that matters is the financial reports. i challenge you to find some equivalent for the medical world. if you have problems with my canadian study then voice them. law2doc i think your denouncement of the validity of the test is a weak argument. the point of the table (accompanied by the entire article and excerpts on page 6 of the thread i believe) is that, contrary to conventional adcom wisdom, mcat verbal scores in particular show themselves to be significantly more accurate predictors of the communication segment of some canadian test (a well established and credible test, the authors note) than the personal interview. essay shows close to no correlation whatsoever.

havarti youve pissed me off in times past so i approach your posts gingerly (out of a sense of responsibility to prevent flame wars), but thanks for heeding my call for evidence--i gtg ill see if i can address it later or maybe someone else can...but i seem to be a one man army here, alas i cant spend all day on sdn fighting interesting (but ultimately useless) forum battles

 

[WholeLottaGame7]

I've had many long and involved arguments with a statistician friend who thinks along much the same lines as you do (though we mostly talk baseball, not medical school admissions). He doesn't want to believe in stuff he cannot quantify, and I refuse to believe that intangibles do not exist. What I've come to realize is that statistics aren't any more REAL than any other factor, because like any other effort they are a human attempt to lay value on an idea/concept/object, etc. It's not like the numbers are inherent to that concept, if that makes any sense.

That said, I am for the most part a big fan of statistics, because it is MORE logical/objective than some other measures. But clearly the MCAT/GPA aren't totally objective. Your score can vary 3-5 points based on getting certain verbal passages or problem sets. You might have a bad day, bad test site, bad etc. I think the MCAT would be better if they averaged you over 10 exams or something to eliminate inter-test variability/reduce the effects of acute stress, etc, but who the hell wants to do that? This is coming from someone who lucked out and scored several points above the average of their practice tests. GPA varies by school, major, etc.

Honestly, in my experience so far, the "numbers" portion of the admissions process has been the random part. I've been rejected from a few schools that, based on my numbers and decisions from other schools, had no business rejecting me pre-interview. That is assuming that schools will interview people they think can succeed in medical school. In contrast, my interviews seem to have been relatively consistent.

As someone who's done some interviewing (to a degree) in the health care field, I can at least sympathize with the interviewers. It would be very odd to not ever meet someone you would be working closely with for the next 4 years (or longer if we're talking about a real job, etc). Regardless of how subjective an interview is, you can still better gauge sincerity, enthusiasm, compassion, etc, in an interview than you can from looking at numbers or essays.

If you're worried about variation in interviewers, etc, there are a variety of ways to address that, as some schools have: increase the number of interviewers, put in place a system to resolve disparities in interview scores, etc.

In sum, I think if you take care of business and apply to a reasonable number of schools, there's no reason a "random" interview should make or break you.

 

[Gut Shot]

Good post, WholeLotta. Nice avatar, as well.

This discussion seems to have devolved to the point where several arguments are being intermingled. Allow me to separate them.

1. Is GPA/MCAT (aka the "objective measures") the best predictor of success in med school and beyond?

2. Do so-called "subjective measures" (maturity, compassion, dedication, etc.) have a place in the admissions process?

3. Is the interview effective at evaluating said subjective measures?

4. If both objective and subjective measures have some utility in the selection process, are they being given appropriate weight relative to their respective values?

These four points are obviously debatable, but I'll give my opinions:

1. GPA and MCAT are the two best tools for ascertaining "what's under the hood." It makes perfect sense that they are the two most important criteria for medical school admission.

2. Absolutely.

3. More often than not. Therefore, despite being variable and frustrating for applicants, I feel they are nonetheless a critical part of the process. The fallibility of interviewers and interviewees is ameliorated by applying to as many schools as possible.

4. Overall I think they're being given appropriate weight. Some schools are number ****** and others give more creedence to intangible qualities. Fortunately for everyone, there are 125 domestic allopathic schools, and folks who play their cards right have a decidedly good shot at getting in somewhere.