Interesting paper on the validity of admissions interviews

[sambone]

http://scholar.lib.vt.edu/ejournals/JVME/V21-2/kelman.html

For anyone who wonders why certain schools don't interview, or wish a school did/did not - here's an interesting paper published by researchers at Cornell Vet School.

(Finding anything I can to procrastinate writing my last OhioSU essay!)

---

Members don't see this ad.

 

[No Imagination]

I hope the adcoms at the schools i'll be interviewing at don't read that. I'm personally counting on my interview to put me over the line

 

[HopefulAg]

*is also counting on the interview*

 

[sumstorm]

I think there may be an issue of methodology here; consider that interviews may weed out problematic candidates that otherwise have good credentials. There wasn't a review of non-admitted applicants. Perhaps that is where the validity of the interview process is important.

 

[LucyLoo]

I have a feeling someone at CSU got a hold of this paper

 

[wildvet]

I definitely see some validity in that paper. I'm actually against interviews if they comprise a majority of the score. There are some schools that place so much emphasis on it, the end result is basically them picking names out of a hat. It's too subjective. One person may interview with a committee that simply doesn't agree with their opinions. Even if someone is older and already a professional, no one is perfect, and just about everyone has their prejudices regarding certain matters. Or maybe the student has associations with the people in the interview (such as a student that went to undergrad at that university and already has connections with people at the vet school). Knowing those on the interview committee will put these people unfairly higher on the list compared to other people that are unlucky and get stuck with interviewers that do not know them.

At the point of the interview, it is obvious that these students are highly qualified and have the experiences as well as the academics to succeed in vet school. Sure, I suppose that the interview may weed out "problematic" individuals, but I'm also sure that the personal statement does enough of that beforehand. So the interview pool essentially just consists of individuals that are all very qualified. At this point, it's all just a luck of the draw.

 

[No Imagination]

Wildvet, while I understand your sentiment, I must disagree with you. Not with the paper; statistically, the conclusions they made appeared sound, but with the importance of impressions of an applicant. Sure, there are prejudices, but, let me ask you this.

Would you hire me as a baby sitter or nanny for your kids based on greatreferences and schooling with a 4 year commitment without ever meeting me?

I've found interviews very helpful in the hiring process, as there are people with 4.0's and 1450 GRE's that have no social skills, leadership qualities, with questionable morals.

As far as LoR, please, we are all computer savvy at some level. Any one of us could have used 3 random people we pulled out of a phone book, made them "[email protected]" and gave the adcoms our friends cell phone #'s.

Just saying, think there is something to be said about impressions, ability to handle yourself under stress, answering tough questions on the spot...

 

[nyanko]

Or maybe the student has associations with the people in the interview (such as a student that went to undergrad at that university and already has connections with people at the vet school). Knowing those on the interview committee will put these people unfairly higher on the list compared to other people that are unlucky and get stuck with interviewers that do not know them.

I'm pretty sure that every effort is made at vet school interviews to prevent this situation from occurring, for what it's worth. Actually, I'm sort of hoping that my MS major professor (a professor in the UCD school of vet med who has been on adcoms in the past) doesn't end up on the adcom for the class of 2014, because otherwise I may not be able to have her write me a rec letter.

 

[LucyLoo]

As far as LoR, please, we are all computer savvy at some level. Any one of us could have used 3 random people we pulled out of a phone book, made them "[email protected]" and gave the adcoms our friends cell phone #'s.

That is just a scary thought...

I agree with you on the importance of meeting someone in person. When I was a hiring manager, I relied on the interview almost as much as the app. It's true some people interview really well and others don't, but you can still get an idea of what that person is about and their level of dedication and enthusiasm.

 

[sumstorm]

Wildvet, there are a lot of people who are excellent writers that have some major problems psychologically.

Also, consider that personal statements may be well-edited. These days people can even hire professionals to conduct major revisions.

I am neither in favor of nor against interviews. I have other issues and concerns about admissions processes that seem unreasonable to me.

 

[No Imagination]

I have other issues and concerns about admissions processes that seem unreasonable to me.

Do share

 

[TrocarKarin]

Wildvet, there are a lot of people who are excellent writers that have some major problems psychologically.

And there are some people who can charm the interviewer's socks off while having some major psychological problems. It goes both ways.

 

[wildvet]

Wow, I really didn't think that my statement would cause such an uproar of opinions lol.

Let's just say that the admissions process is imperfect in general. It's obvious at that, or else they wouldn't have done studies like that in the first place. I've heard of papers that have found opposite results too (which is the reason why Illinois was compelled to make their interview heavily weighted). Some people will slip through the cracks of the interview; it is not perfect. If people are smart enough to get past the academic review, they might be smart enough to pull off a very good acting game by faking their way through the interview and into vet school. You can't deny that there are people like that out there.

 

[No Imagination]

Wildvet, I should also say, I would typically agree with you completely, that objectivity is much more desirable that subjectivity. Truth is, i'm still bored, and looking for some intellectual debate wherever I can find it So, everyone voting for McCain? <-- I'm JUST KIDDING!

 

[LucyLoo]

<-- I'm JUST KIDDING![/B]

Wow, you are desperate!!!

 

[Catnapper]

And there are some people who can charm the interviewer's socks off while having some major psychological problems. It goes both ways.

I agree with both of you. In addition, I'd have to say that the same might hold for those who live up to the stats Cornell relies upon. I'm sure there are plenty of perfect scores on paper who can't function as people (or with people--which would seem critical in clinical practice!).

 

[rachroo]

Would you hire me as a baby sitter or nanny for your kids based on great references and schooling with a 4 year commitment without ever meeting me?

I've found interviews very helpful in the hiring process, as there are people with 4.0's and 1450 GRE's that have no social skills, leadership qualities, with questionable morals.

I'm sure there are plenty of perfect scores on paper who can't function as people (or with people--which would seem critical in clinical practice!).

And there are some people who can charm the interviewer's socks off while having some major psychological problems.

WOW.

Those statments pretty much sum up my entire reasoning on the topic....
I personally think that interviews are (or should be) a very important process for one being accepted into vet school (even if someone doesn't plan on going into clinical practice).

 

[projekt]

So, now that we know that GRE scores have, at best, a slight positive correlation with academic performance, and interviews probably have no correlation with clinical performance, how do we decide who to put into vet school? Since vet school is generally state-supported, perhaps applicants should make a business case for why they deserve the spot and how they will be a net benefit to the state J/K

Or, perhaps they should require evaluated internships during the preparatory curriculum, to see how people actually do perform in their area of interest.

 

[Bill59]

The first part of the study is interesting, where they found poor correlation between admission interview score and rating the same characteristics by clinical faculty in the 4th year. I think it would have been more useful to compare interview score with clinical performance score though, because that's more in line with what the admission process is intended to accomplish.

The second part of the study is mostly useless. They compared scores from a class that was interviewed to a class that was not interviewed. They found a trend toward a a better score for the class that was interviewed. Since this difference was not statistically significant, we don't know whether (1) there is truly no difference or (2) there is a true difference but the study was not powerful enough to detect this difference. They should have provided a power analysis so readers could judge which of these two possibilities is more likely.

Of course this study is 14 years old so I suspect there is more recent studies on this.

 

[nyanko]

Or, perhaps they should require evaluated internships during the preparatory curriculum, to see how people actually do perform in their area of interest.

This seems like it would be an unmitigated logistical nightmare.

I mean, in theory, sure, but can you imagine coordinating this sort of thing across thousands of applicants yearly? Different schools would want to look at different things, and what about people who decide to apply to school X at the last moment? And nontrads? WOW..

 

[projekt]

Since this difference was not statistically significant, we don't know whether (1) there is truly no difference or (2) there is a true difference but the study was not powerful enough to detect this difference.

It would be good, from a scientific POV, to alternate years between interviewing and not interviewing for like 6 years. Then they'd have a better idea of the stability of the result.

There is also the potential problem that the study was intended to justify their choice to eliminate interviews after the fact, so it may have been designed to be more likely equivocal in the second area.

 

[projekt]

This seems like it would be an unmitigated logistical nightmare.

The only way to know is to try it. Avoiding "logistical nightmare" is the typical excuse given for using the SAT or GRE, or coming up with a way to compare grades (not GPAs) at different schools. Cutting corners may generally shortchange applicants. (How much? Which applicants? Who knows?)

Luckily, it seems that around 95% of admitted DVM candidates are able to graduate. Some years it's more (and it causes alarm, even though statistics would say it shouldn't) and some less. That means that even if someone who would have graduated gets rejected by the admissions process, it's probably true that the person who was accepted in their place also graduated.

 

[projekt]

I think it would have been more useful to compare interview score with clinical performance score though, because that's more in line with what the admission process is intended to accomplish.

I think what was shown is that the interview and its short time frame distorts the view of qualities that are evaluated more closely in clinicals over a longer period of time. That, in and of itself, should mean that the interview is unreliable. Whether the interview accidentally discovers other qualities that are not on the evaluation form, which contribute to success in clinicals, we will probably never know. That's sort of like saying that, while the treatment doesn't interact with the disease in vitro, nevertheless it may have some unknown effect that wasn't tested for. Possibly true, but where is the scientific basis?

 

[No Imagination]

That's sort of like saying that, while the treatment doesn't interact with the disease in vitro, nevertheless it may have some unknown effect that wasn't tested for. Possibly true, but where is the scientific basis?

Not sure I am following your logic here. Sure a bit off tract, but you said “Treatment”, so there MUST be some reason to assume it has some effect (primary or secondary) on the disease, otherwise it wouldn’t be a treatment. If it there is not a direct influence on the disease in vitro, then surely, but virtue of it being categorized as a treatment, in vivo, one would expect that some response to the challenge. No scientific basis, but logic.

Same thing with interviews and candidates (see what I did there

 

[HopefulAg]

It would be good, from a scientific POV, to alternate years between interviewing and not interviewing for like 6 years. Then they'd have a better idea of the stability of the result.

Yah but this is people's lives and futures you're playing with. Not something you can just play around with to see what works for some and what shafts others.

 

[david594]

Yah but this is people's lives and futures you're playing with. Not something you can just play around with to see what works for some and what shafts others.

So is the sentiment of your statement that by not interviewing they might let in applicants who are unfit?(academically or emotionally?) If they make it to interview, we can assume that they were at least deemed academically qualified. Is it the vet schools responsibility to ensure the applicants "really want it" enough?

 

[sambone]

Would you hire me as a baby sitter or nanny for your kids based on greatreferences and schooling with a 4 year commitment without ever meeting me?

I've found interviews very helpful in the hiring process, as there are people with 4.0's and 1450 GRE's that have no social skills, leadership qualities, with questionable morals.

This is an old quote above, but an interview I heard the other day made me think about this. The news org was interviewing a surgeon who had done some amazing surgery, very successfully -- yet he sounded like the kind of guy who would NOT do very well in an interview - difficult to talk to, not the best orator/conversationalist and possibly no social skills. While it's really important to communicate well as any kind of physician, I think in the end, if I had to choose someone to do my surgery I'd go with the person who has better stats on paper over the person who is articulate and easy to talk to. (Not that they are mutually exclusive - ideally they'd be great at both!) I just don't think we can compare choosing future doctors/scientists to choosing babysitters.

 

[HopefulAg]

So is the sentiment of your statement that by not interviewing they might let in applicants who are unfit?(academically or emotionally?) If they make it to interview, we can assume that they were at least deemed academically qualified. Is it the vet schools responsibility to ensure the applicants "really want it" enough?

I'm not sure I follow.

I think interviews are a good idea. And yes if they make the interview they're academically qualified (in the schools' eyes) for that position. The interview allows for the school to see those who aren't emotionally qualified (IE applicants who come stoned to the interview, for instance. Yes it's happened).


The problem with that comes into different caliber of schools.


And no it's not the vet schools responsibility to insure the applicants 'really want it'. Where'd you get that?


All I said is that radically changing the process every year isn't a very good idea because you're messing with people's futures.

 

[david594]

And no it's not the vet schools responsibility to insure the applicants 'really want it'. Where'd you get that?

I couldn't come up for a better example to describe those who are emotionally qualified. People who know its about more than just puppies and kittens, or that we can't help every animal, etc(not showing up stoned I guess would fall in this category).

All I said is that radically changing the process every year isn't a very good idea because you're messing with people's futures.

The study cited indicated that there was little difference in the individuals graduated. So while this might seem like a radical change in one part of the process, it would appear to have little impact on the end result.


Now when it comes to being concerned about messing with peoples futures, I would be more worried about the Caribbean veterinary schools. Their attrition rates would indicate that they are accepting students who are not academically qualified just to let the system sort them out.


It seems like Interviewing in general may be a poor mechanism for selecting people, but still has value in being used for eliminating others.

 

[chris03333]

This is an old quote above, but an interview I heard the other day made me think about this. The news org was interviewing a surgeon who had done some amazing surgery, very successfully -- yet he sounded like the kind of guy who would NOT do very well in an interview - difficult to talk to, not the best orator/conversationalist and possibly no social skills.

That is pretty normal stereotypical surgeon personality...

 

[chris03333]

The study cited indicated that there was little difference in the individuals graduated.

Yes but those individuals who graduate are very very different than they were when they were accepted. Vet/Med School chages you...

 

[ri23]

The study is definitely flawed in some ways.

First of all, the interview is not designed to score and test clinical knowledge. While that is certainly something that schools do look at it is only a portion of the purpose of the interview. Schools are much more likely to accept a student that may not be able to score well in a clinical knowledge section yet, but that the interviewers feel are capable of learning the material than a student that has a 4.0 but seems unable to cope with stress (school IS stressful).

Also, while Ohio State weights the interview 55% this includes references, personal statement, strength of academic program, etc.

I think interviews are essential, they give you a lot of information that can be reliably gotten from an application (people can get a lot of help with essays). Most schools take a lot of precautions to ensure equality in interviews and have a standard to reduce bias.

 

[sumstorm]

Earlier in the conversation, No Imagination asked what I consider greater concerns, so I will toss those out:

I find it very puzzling that being related to a veterinarian OR a faculty member of a school can impact you admissions. If it couldn't, applications wouldn't ask for this information.

I might be biased, though, because if GPA/GRE algorithms were the sole or even primary factors admissions were based on, I wouldn't have a chance. I made a 3.4 including graduating with honors research and have a 1340 GRE combined. I also worked a full time job while attending school full time, and at times worked up to 60 hours a week outside of school. If schools didn't consider things like work, research, post-graduate jobs, and other experience, I probably wouldn't stand a chance.

 

[rachroo]

Yes but those individuals who graduate are very very different than they were when they were accepted. Vet/Med School chages you...

Hey chris03333, could you expound on that? I'm interested in how and why (besides hopefully becoming a better clinician, surgeon, etc).

 

[twelvetigers]

Are you the same person you were when you started undergraduate? Imagine yourself fresh out of high school, jumping into the college experience. I bet you've changed - it'd be hard not to. Vet school will be the same, only more dramatic because of the increased stress, very different situations, possible tensions in relationships, and maybe some friends that are much closer to you than any have been before.

Not to answer for anyone, since I know you asked Chris. Just tossing in my .02!

 

[NoRSquared]

Hi All -- This is a very interesting discussion to read, esp. because my husband and I talk about it quite often. He is ass't dean of admission at an undergrad institution that does not do any interviews. One thing that hasn't been addressed on this thread at all (and it is one of the major reasons why his school decided not to interview) is the disproportionate burden that interviews place on low-income applicants. As some of you probably know first-hand, the application fees alone are a financial strain; add to that the cost of traveling to interviews, and the process becomes financially impossible for some. The school where my husband works found that many low-income applicants simply do not apply to schools that require interviews, or they are forced to turn down interview offers and place their bets on only a couple of schools where they can afford to interview. This creates a real imbalance in the application process and limits the chances of application success for low-income applicants. Simply put, the school decided to forego interviews to cut out this imbalance altogether.

I know this is a completely separate reason for getting rid of interviews than has been discussed in this thread, but I thought it might be interesting to think about. Good luck to everyone applying this cycle!

 

[twelvetigers]

I hadn't even thought of that. It's definitely a factor to be considered!

 

[charismarl]

Wow that is really interesting! Boo to interviews! I think it's way more stressful than it needs to be...you hang the whole rest of your life basically, in a way, on a 15 minute interview. If there really isn't a positive correlation between interviews and success later on then it seems rediculous to waste people's resources on it. I'm sure more studies will be done and I'm sure some schools will never change, but I'm interested to see what happens in the future...

 

[chris03333]

Hey chris03333, could you expound on that? I'm interested in how and why (besides hopefully becoming a better clinician, surgeon, etc).

what twelve said is a decent way of explaining...kinda hard to explain but you will understand what I mean once you have been in vet school for a while.

 

[projekt]

Not sure I am following your logic here. Sure a bit off tract, but you said "Treatment", so there MUST be some reason to assume it has some effect (primary or secondary) on the disease, otherwise it wouldn't be a treatment. If it there is not a direct influence on the disease in vitro, then surely, but virtue of it being categorized as a treatment, in vivo, one would expect that some response to the challenge. No scientific basis, but logic.

Well, the short answer is, no.

A treatment is anything that someone thinks might help a disease condition. Generally, studies have a "treatment" group and a "control" group. The existence of those two categories is not a statement that the "treatment" group already works. Quite the opposite, actually.

The reason why I used the "in vitro" thing is because we tend to study drugs and the like in cell culture because we want to show that we understand the effect it has. We hypothesize about this effect. It's not really ethical to go to human studies until we find that there is an in vitro effect, even if some hypothesize that there is an effect that may occur that we haven't studied.

 

[No Imagination]

Well, the short answer is, no.

A treatment is anything that someone thinks might help a disease condition. Generally, studies have a "treatment" group and a "control" group. The existence of those two categories is not a statement that the "treatment" group already works. Quite the opposite, actually.

I will concede to that logic, esp to the definition of treatment in a trial.

However, if

while the treatment doesn't interact with the disease in vitro, nevertheless it may have some unknown effect that wasn't tested for. Possibly true, but where is the scientific basis?

If there is no scientific basis, then why is it considered unethical to move to stage 3 clinical trials (on humans) if the in vitro was not performed, seeing as, regardless of the results, they may have no correlation with the in vivo findings.

I'm agreeing with you in that in vitro findings don't necessarily correlate with in vivo, but I believe there is a scientific basis to extrapolating the results for further study. It's not just done to appease ethics committee's.

Ahh, I think I am just going to let this one go, as I think this debate took a turn towards the symantics, and thats kinda boring. Good post though.

 

[chris03333]

I will concede to that logic, esp to the definition of treatment in a trial.

However, if



If there is no scientific basis, then why is it considered unethical to move to stage 3 clinical trials (on humans) if the in vitro was not performed, seeing as, regardless of the results, they may have no correlation with the in vivo findings.

I'm agreeing with you in that in vitro findings don't necessarily correlate with in vivo, but I believe there is a scientific basis to extrapolating the results for further study. It's not just done to appease ethics committee's.

Ahh, I think I am just going to let this one go, as I think this debate took a turn towards the symantics, and thats kinda boring. Good post though.

Drugs don't go from in vitro to stage 3 clinical. You have to go into animals (usually rodent, then higher species first) before you even do any human trials

 

[sumstorm]

The undergrads I applied to required interviews, but had interviewers in major cities across the country...and would arrange to meet at a reasonable location for the applicant.

Considering that vet school is very expensive, not sure the interview costs would make much of a difference.

 

[VAgirl]

Yes but those individuals who graduate are very very different than they were when they were accepted. Vet/Med School chages you...

I know this is off-topic/tangential, but I would speculate that for most vet students, a substantial portion of the change they go through can be associated with the life stage they're in. 22-26 is a time when most people go through a lot of changes (evaluating where they are, maturing, etc.) I know I went through huge changes/personal revelations during this time, and I wasn't a vet student yet.

 

[HopefulAg]

The study cited indicated that there was little difference in the individuals graduated. So while this might seem like a radical change in one part of the process, it would appear to have little impact on the end result.

While it is possible that that is true, if there's one thing I learned from my statistics class it's to take studies with a grain of salt. Most anything can be 'proven' if you design the experiment correctly. And this is only one study. Done on a very small population (78 people) and the results are subjective to the interviewer.

I'd believe it if there's been hundreds of studies done and all come to the same conclusion, but just this one? Nothing should be radically changed based on one study by one group on one group of individuals. In addition the study is from 1992. That's a 16 year old study. The veterinary profession, admissions process, and applicant pool has changed during that time.

 

[runnerDC]

Actually, I think this is right-on. The age of most students in vet school happens to coincide with the age during which many people go through a lot of personal growth. I don't think it's vet school itself that changes people as much as it is where they are in life, in general, and what they are figuring out about themselves during that time.

I know this is off-topic/tangential, but I would speculate that for most vet students, a substantial portion of the change they go through can be associated with the life stage they're in. 22-26 is a time when most people go through a lot of changes (evaluating where they are, maturing, etc.) I know I went through huge changes/personal revelations during this time, and I wasn't a vet student yet.

 

[chris03333]

I know this is off-topic/tangential, but I would speculate that for most vet students, a substantial portion of the change they go through can be associated with the life stage they're in. 22-26 is a time when most people go through a lot of changes (evaluating where they are, maturing, etc.) I know I went through huge changes/personal revelations during this time, and I wasn't a vet student yet.

And I would argue that you will change substantially because of your vet school experiences now, that have nothing to do with your life changes then.

You can always come back in 4 years and update this thread

 

[twelvetigers]

I hope there is never a point in my life that I don't change over the span of four years.

 

[VAgirl]

And I would argue that you will change substantially because of your vet school experiences now, that have nothing to do with your life changes then.

Undoubtedly there will be changes. There always are.

 

[projekt]

Most anything can be 'proven' if you design the experiment correctly. And this is only one study. Done on a very small population (78 people) and the results are subjective to the interviewer.

I'd believe it if there's been hundreds of studies done and all come to the same conclusion, but just this one?

I agree with you in part. But, on the other hand, interviews were adopted as a practice long before anyone did any study to see if they were useful.

I'm finding this thread interesting mostly because it is showing us that we have a long way to come with evidence/science-based medicine. There is a lot of untested dogma out there. Why does nearly every (American) veterinarian do spays at 6 months? Why do they do a full OHE, and not just an OE? Even in the face of data showing that spays are easier for patients when done at younger ages, vets are not changing their practice.

It took a long time for pain meds to be used in vet clinics. Virtually every older vet who comes through our school jokes about the time, around late 80s early 90s, when animals started to feel pain after surgery.

Our school currently eschews interviews and it seems to draw a talented and diverse class, probably more diverse than it did when it interviewed, but I haven't run the numbers. I also mean diversity in terms of interest area, not just race/religion/etc. Last year, six people dropped out rather than the two expected, but that's probably not beyond statistical limits. (Unfortunately, the administration is acting like it was a big problem, so they're probably overcompensating this year.)