Reliance on Interviews in Clinical Psychology

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
Joined
Aug 31, 2011
Messages
2,514
Reaction score
4,388
The subject of interviews came up in another thread. I've always been curious why clinical psychology is so reliant on interviews for making determinations regarding future performance. The empirical literature on interviews is pretty bad, with no consistent finding (at least that i am aware of) showing that they have good predictive validity, yet die hard empiricist will still insist that they are valuable. Scores from highly structured interviews might have some incremental validity when combined with other data (e.g. measures of things like cognitive ability and conscientiousness), but how many of you use highly structured interviews? Seems to me that 100% of students or employees that didn't work out made it through an interview!

Not sure what my point or question is here- maybe I'm just venting! Do you guys feel that interviews give you valid information about candidates abilities to perform in the role they are being interviewed for? If "yes", than what is your basis for that conclusion? Bonus points- provide and empirical reference in support of any "yes" answers!
 
I assume you mean interviews for internship/postdoc/staff positions?

If so, I don't rely on the interview. The application and background make most of my decision for me, and checking references. I rely on past performance. The interview is maybe 5-10% of my decision making process I'd estimate. Helps to weed out some obvious axis II.
 
Clarification: Sounds like you are talking about "job" interviews, but want to make sure since its obviously different if referencing clinical interviews or other interviewing contexts.

I'm actually starting to see pushback against relying on them very heavily in the settings I have been attached to...they are used but are mostly a minor consideration. For me, its mostly: A) An opportunity for me to rule out anyone who looks good on paper and does something outright insane (which...has happened); B) An opportunity to gauge personality fit. This matters. I have definitely worked with people who I would likely have to give high performance ratings (depending on the scale used) that I would never, ever, ever want to work with again. For better or worse, I think this is the main reason they are used. Most people would not feel comfortable signing up to spend 40+ hours per week for an indefinite time period with someone they had never met. And unless you are an undergrad, if you can't spitball grant ideas with me on the fly, I have zero interest in working with you.

I also think it depends what context we are referencing. Faculty/post-doc interviews are not usually meant to gauge "is this person going to be productive" - it is an opportunity to learn more about someone's visions for their research program in a way you could never get from a CV or research statement, how it might dovetail with department goals and an initial opportunity for them to meet people and start talking collaborations (arguably more for the candidate than the school). The job talk isn't going to tell me if this person will be successful. Its going to tell me if this person will be a good collaborator with X, who I know is kind of stuck on an island right now given their direction, etc. I don't know how one would operationalize that in a measure.

Also - for what its worth - the vast majority of research showing interview have limited predictive potential were not optimally designed, so factors like measurement variance, etc. can account for a lot of the issues. I don't recall seeing many RCTs of it - a lot of it was epi-style studies looking at variance accounted for by factors. Which is valuable, but not definitive. Most of the literature also (understandably) focuses on different types of roles and trends towards lower level positions. I think the interview is much less important for hiring a technician. Probably more important for a department chair.
 
In all fairness, I attended open houses for internship apps (which they specifically offered instead of interviews for the reasons discussed above) and I hated them more than actual interviews. A lot of people would try to show off their knowledge in other ways, like asking certain questions, and I found that aggravating. I also am an introvert so having to go up and talk to people all day was not my idea of fun.

I mean, I agree that interviews suck and wonder why we're using them based on this research, but the open house idea seems to have its flaws too. Not sure what the solution would be.
 
Legally, you’d have to define every aspect of the job and create objective metrics that do not show group differences on gender or race. No one wants to do that. We want to like our coworkers.
 
Not sure what my point or question is here- maybe I'm just venting! Do you guys feel that interviews give you valid information about candidates abilities to perform in the role they are being interviewed for?

Not really, but of course we're not just selecting candidates based on their ability to perform the essential duties of a position.

Helps to weed out some obvious axis II.

An opportunity for me to rule out anyone who looks good on paper and does something outright insane

We want to like our coworkers.

A lot of people would try to show off their knowledge in other ways, like asking certain questions, and I found that aggravating.

All of this. The interview is often a rule-out or a tiebreaker.
 
The subject of interviews came up in another thread. I've always been curious why clinical psychology is so reliant on interviews for making determinations regarding future performance. The empirical literature on interviews is pretty bad, with no consistent finding (at least that i am aware of) showing that they have good predictive validity, yet die hard empiricist will still insist that they are valuable. Scores from highly structured interviews might have some incremental validity when combined with other data (e.g. measures of things like cognitive ability and conscientiousness), but how many of you use highly structured interviews? Seems to me that 100% of students or employees that didn't work out made it through an interview!

Not sure what my point or question is here- maybe I'm just venting! Do you guys feel that interviews give you valid information about candidates abilities to perform in the role they are being interviewed for? If "yes", than what is your basis for that conclusion? Bonus points- provide and empirical reference in support of any "yes" answers!
A picture is worth a thousand words, so to speak and, perhaps, a moving interactive picture in context that must respond to unpredictable stimuli/questions is even more informative. I think an interview situation allows the interviewer to get a real sampling of interpersonal style/fluidity (a 'behavioral sample,' if you will) that may either be consistent or inconsistent with descriptions (or self-descriptions) of behavior. The fact that it is a brief slice of behavior should be taken into account but it can be a useful source of information.
 
Additionally, as I've said before, the interview is probably most important for the applicant. Getting to see the actual site and get a feel for everything is pretty key. My rankings for places changed dramatically after onsite interviews.
If everyone at the site was presenting things in an honest manner then yes this would be helpful to the applicant. As an applicant I've experienced enough lack of honesty from a site (whether student interview or job) to know I'm never going to get the full story so I'm with camp interviews are a waste of time.
 
If everyone at the site was presenting things in an honest manner then yes this would be helpful to the applicant. As an applicant I've experienced enough lack of honesty from a site (whether student interview or job) to know I'm never going to get the full story so I'm with camp interviews are a waste of time.

YMMV, do with that whatever you want.
 
I agree that it's desirable to identify traits in candidates that would not gel well with ourselves, our labs, our departments, etc. I also get the desire to identify and avoid "axis II" stuff (though I think we overuse that term colloquially). As does PsyDr, I also want to like my coworkers.

I offer this unpopular opinion: We are not as good as we think at using interview data to identify problematic traits, "Axis II" stuff, and people we would or wouldn't get along with. Further, I'd suggest that are perceptions of our abilities to do so are biased by the following:
a) by the time a candidate gets to the interview phase they are probably a good match and are more likely to do well than not do well
b) most of us are generally agreeable and likable people with a history of learning and working in complex settings, which has resulted in a pretty high likelihood that we are going to get along with most people anyways (this one probably doesn't apply to PsyDr 😉)
c) we discount our failures (e.g. people we interviewed and turned out to have undesirable traits, axis II stuff, or were generally unlikable), often times retroactively giving ourselves credit for identifying these factors during the interview even when we didn't (e.g, "I KNEW something wasn't right about him!")
d) when our decisions work out well, we attribute it to our innate and internal abilities to judge character and predict behavior, rather than to a + b above.

I'd further offer that typical interviews don't actually improve our outcomes, and we may actually be missing out on good candidates by over-weighing factors not related to the candidates ability to perform in the position.

Oh well- ultimately it's an empirical question. Anybody want to be in the experimental group that stops doing interviews?

(I do agree with many of Ollie123's points)
 
Last edited:
@ClinicalABA

I'm actually a pretty likeable person. Facilities, professional orgs, academic institutions, etc, like to have me for some reason. However, I'm in an unique position where I don't have to put up with a lot of things.
 
@ClinicalABA

I'm actually a pretty likeable person. Facilities, professional orgs, academic institutions, etc, like to have me for some reason. However, I'm in an unique position where I don't have to put up with a lot of things.
At least that's what you say. I'd have to interview if I wanted to be certain.
Also- that's not really a unique position, but one that should be the well-earned norm for a well-trained, effective, efficient, and productive professional.
 
I've been thinking about this too, as one part of a greater set of thoughts about graduate school admission in clinical psychology. I dislike using the GRE for admission (though my program isn't going to stop anytime soon) in part because it is not particularly predictive and is problematic for underrepresented groups. I generally suspect that there are a lot people applying to clinical psych programs who COULD be great grad students and clinical psychologists, but didn't get good mentoring on how to apply, or didn't have the research opportunities to be competitive. I'm not sure of the solutions to these problems but they kinda bum me out around application time every year.

As for interviews, I find them really important for personality fit, as Ollie said above. For grad students, I'm going to be working with this person for 5+ years and I want to like them, enjoy seeing them in my office, lab meetings, and in the clinic for the entire time of grad school. Am I going to figure all that out in two days? Probably not, but I'll get a clue. If the applicant is rude to my students or other applicants but nice to me (which has happened), I don't want to work with that person. Interviews give me a chance to see how a person thinks on the fly, interacts socially, etc. Do I have to take pains to try to see past the anxiety/nerves/impression management? Yes. Do I try to ask questions to get people to reveal their thinking processes and work style? Yep. So far I've really like all the grad students I've admitted, and the interviews were always major pieces of the puzzle.
 
Oh well- ultimately it's an empirical question. Anybody want to be in the experimental group that stops doing interviews?
Texas Women's University's Counseling Psych PhD program actually did this for a couple of years in the interest of limiting the impact of implicit bias, but then went back to doing interviews.
 
Interviews are very important..we are very social beings..you want to enjoy work and like/be friends with your colleagues..do I want a coworker that is smarter/more capable/boring/a bit weird or a bit less capable but willing to learn and also fun and great to hang around..you get the idea
 
Research for employment interviews, in general (not specific to admissions to training program).

A recent meta-analytic review of 111 studies, however, shows that the reliability of the interview varies widely depending on the situation. Conway, Jako, and Goodman (1995) found that the average reliability of highly structured individual interviews was 0.59, while the reliability of unstructured individual interviews was 0.37. As might be expected, reliability was much higher for panel interviews. Surveys reveal that in practice, unstructured interviews are much more likely to be used than structured and, in this case, the reliability of the average interview is likely to be close to 0.37.

Validity of the interview would be even lower than that.
 
Currently, I'm a grad student applying to internships. For me, I don't understand the confusion, and think the question is being asked possibly out of frustration with the state of things and the whole interview process? (I feel that, for sure) That said, actuarial supremacy of sorts has been around for decades, but we are human beings. Assuming I land interviews, I'm going to be listening to my gut at these places. I'm going to evaluate the supervisors' questions, demeanor, degree of stuffiness, etc. I also want to see the interns' faces to see if they appear they've lost the will to live - something humans can do much better than algorithms. I'm also intrigued to see if supervisors themselves know the difference between multidisciplinary and transdisciplinary, and to see what environment they're offering. I feel like questioning interview utility in the way you are doing may be too reductionistic. What kinds of "future performance" are you referencing, and how is that "objectively" being measured?
 
Currently, I'm a grad student applying to internships. For me, I don't understand the confusion, and think the question is being asked possibly out of frustration with the state of things and the whole interview process? (I feel that, for sure)

Nope! I'm well past internships, and am really hoping to not have to apply for any position ever again! I was prompted to ask the question by another thread on the topic of skype/video interviews. I am familiar with the research on the validity of interviews, and the finding are, at best, equivocal, trending more towards a result of "we're not really good at getting or using valid data from interviews" (though as others have posted above, the research methods and analogs to actual interview formats aren't great). The notion of using newer technologies in interviewing got me thinking about the research related to existing methods and wondering how it my apply.

That said, actuarial supremacy of sorts has been around for decades, but we are human beings. Assuming I land interviews, I'm going to be listening to my gut at these places. I'm going to evaluate the supervisors' questions, demeanor, degree of stuffiness, etc.

These are things that i really want to be able to do and think I can do, however there is not strong research evidence that we're particularly good at such things. In fact, we tend to be distracted by irrelevant data (such as physical appearance; outward expression of gender; even the relationship between how people look and how they smell!).

I also want to see the interns' faces to see if they appear they've lost the will to live - something humans can do much better than algorithms.

This sounds like hyperbole, but citation?

I'm also intrigued to see if supervisors themselves know the difference between multidisciplinary and transdisciplinary, and to see what environment they're offering.

I do think that interviews are more important for the interviewee, and MAY yield more valid data (but i'm not sure) for accurately predicting if the place is a good fit or not. As far as knowing the difference between "multidisciplinary" and "transidisciplinary," do you really need and interview to determine that. Also- here's some unsolicited advice, so take it for what it's worth- I'd caution you about how much, as an intern "interviewee" you pursue this line of questioning. It sounds like it's an very meaningful distinction to you, but might not be so to most others AT THE PRACTICAL LEVEL OF INTERNSHIP SUPERVISION (as opposed to the philosophical or system level). It could be an off-putting discussion for many supervisors

I feel like questioning interview utility in the way you are doing may be too reductionistic.
Maybe, but "reductionistic" is not universally pejorative (and it's pretty cliche to be going around calling a behavior analyst reductionistic, don't you think! Next you be accusing me of turning my clients into robots! 😉 note: this winking emoji signifies that what directly preceded it was sarcasm). I think my original question was actually not reductionstic, but whatever- it's kind of a strawman anyways.

What kinds of "future performance" are you referencing, and how is that "objectively" being measured?
I am interested in whether or not candidates can meet the job requirements, get along reasonably well with co-workers, generally come to work prepared, seek supervision and training when necessary, use that supervision and training constructively, and generally advance the mission of the agency. I'd also like to know whether or not they are going to do crazy things like steal stuff, lie, inappropriately hook up with coworkers, engage in damaging gossip, etc. As an added bonus, I'd like to know if they would be fun to have come to trivia with our group from work, offer knowledge in an area of relative weakness, and engage in pleasant conversation over a fine beverage. If they are interns or other trainees, I'd also want to be sure that they clearly understood and desired the type of training experience we could offer, and would like to know if they were going to be able to represent us well when their training was through. In short, probably the same as you and the others who have posted above. I guess I'm just a little more skeptical of our abilities to validly solicit this information through our standard interviews, and would again offer that i think we discount or failures and over-rate our successes.
 
Last edited:
Admittedly don't know the data here, but it's much more common for the grad programs I know of to use interviews to bump up impressive applicants and bump down red flag applicants than to guide the entire ranking list.

If that's the indeed case, I'd be interested in how interview impressions of something like > 90th percentile vs. < 10th percentile ratings of applicants relates to performance in graduate school. Most of the studies I've seen analyze purely linearly, but again that's not my understanding of how most programs actually use grad school interviews.

In clinical psych in particular, where grad students are going to be responsible for client/patient care, interpersonal skills matter. That doesn't necessarily mean that would have to be assessed during an interview (let alone if we can do it at all), but we would need establish some sort of metric for this that application materials are unlikely to cover.
 
Last edited:
Top