Relevant performance-based interview ideas for grad school

[MCParent]

---

Members don't see this ad.

Hi, all.
I'm considering using performance-based interview tasks for this round of interviews (both in pre-live interviews, tasks conducted online, and on interview day). I'm having a bit of trouble figuring out exactly what the tasks might be though, without having them feel really dry. I don't expect incoming doc students to demonstrate sophisticated clinical thinking or report writing. I also don't want to test on things that I see as pretty trainable (e.g., how I like research articles to be written). Hoping for some brain stormed ideas!

 

[WisNeuro]

How about a research design writing task. Given a set of parameters, can they think of a way to feasibly test a hypothesis? Would get at reasoning and problem solving ability, as well as how well they can think through a process from beginning to end.

 

[Seven_Costanza]

Hi, all.
I'm considering using performance-based interview tasks for this round of interviews (both in pre-live interviews, tasks conducted online, and on interview day). I'm having a bit of trouble figuring out exactly what the tasks might be though, without having them feel really dry. I don't expect incoming doc students to demonstrate sophisticated clinical thinking or report writing. I also don't want to test on things that I see as pretty trainable (e.g., how I like research articles to be written). Hoping for some brain stormed ideas!

At one program interview, I was given a moderately complicated graph and was asked to interpret it.

 

[DynamicDidactic]

I am curious what you find to be lacking in the traditional interview model? Or more precisely, what can be improved? This would help clarify the tasks. Are you looking for higher intelligence, more motivation, social ability, creativity, scientific acumen (or perspective acumen), resilience (grad school can be hard), social justice perspectives, retention, etc?

 

[futureapppsy2]

At one program interview, I was given a moderately complicated graph and was asked to interpret it.

Was this an ABA-focused program? I can think of a lot of areas in psychology where one could work at a high level for years and never need to interpret a moderately complicated graph (my very, very behavior analytically-focused program was not one of them, however! 😉 )

 

[Therapist4Chnge]

Great idea. I should recommend this to my friend who has had a hard time with “great on paper, talked a good game about research and clinical goals, but are very underwhelming in regard to their actual (research) work once in the door.”

 

[Justanothergrad]

Ask them to critique the validity of the GRE at predicting graduate school success via a research design. Its cathartic as hell and tackles research analytic approaches.

I've typically asked them to critique an existing paper (its not a popular part of my interview), but I may switch to this design this year instead as I think about it. Same outcome information for me.

 

[MCParent]

I am curious what you find to be lacking in the traditional interview model? Or more precisely, what can be improved? This would help clarify the tasks. Are you looking for higher intelligence, more motivation, social ability, creativity, scientific acumen (or perspective acumen), resilience (grad school can be hard), social justice perspectives, retention, etc?

I just want better data than “your research is good and I want to be a good researcher.” 🙂

I was thinking knowledge but if you have ideas to assess those other things via tasks I’m all ears. I’m saying my challenge is coming up with good, short-ish tasks for these things.

@Justanothergrad you think it would cause psychic trauma to make them critique one of my own papers?

 

[str63]

@Justanothergrad you think it would cause psychic trauma to make them critique one of my own papers?

I think if you have them critique your paper, you'll only get them saying positive things - then you won't know if they're just trying to be nice, suck up, or if they can't identify limitations. Or, if they actually do provide critiques on your paper, I wouldn't know if they were assertive/brave or had personality stuff going on. 🙂

When I was interviewing for grad school years ago, I had someone ask me to design a study of my choice, which I really liked. It gave me the opportunity to show what I was interested in, as well as provide knowledge regarding research design. And it took it one step further then me just saying "I like your research."

 

[DynamicDidactic]

I just want better data than “your research is good and I want to be a good researcher.” 🙂

I was thinking knowledge but if you have ideas to assess those other things via tasks I’m all ears. I’m saying my challenge is coming up with good, short-ish tasks for these things.

When I've been interviewed I always found these sort of tasks to be poor predictors (not that I came across a lot of these). Pre-grad school I interviewed for a RA position and was asked to discuss the difference between within-subject and between-subject variance. From my current perspective, a seemingly worthwhile question to elucidate an applicants statistical acumen. I came home, checked up the answer, kicked myself in the head for getting it wrong, and fully remember the difference at that point. Still got the job and I was very good at it. At my previous faculty position, we required master's-level applicants to provide a writing sample when they showed up for group interviews (they didn't know this in advance). I didn't find, anecdotally, that the writing sample provided any additional predictive validity for how well the students would do in the program. I remember in some grad programs and internship interviews where similar questions were given.

If students are not expecting a performance-based interview, I do not think you will get useful data. If you told them in advance that you will ask them to critique a paper of yours (or something else), that is a different set up. I would rather know how well someone performs when they know what the task is going to be (I don't want to surprise you with a sprinting race when you have been training for a marathon). For interviews, I want to see that you prepare well for an interview (e.g., socially approachable, can articulate interests and past experiences). If I want to get a feel for their knowledge base, I'd tell them in advance that you will ask them certain questions (or tasks) to assess their knowledge in a certain area (e.g., methodology, statistics, your research area).

Frankly, I don't think the current method of choosing doctoral grad students is great but I don't know if there is a more valid approach. Just my 2 cents.

 

[Justanothergrad]

I just want better data than “your research is good and I want to be a good researcher.” 🙂

I was thinking knowledge but if you have ideas to assess those other things via tasks I’m all ears. I’m saying my challenge is coming up with good, short-ish tasks for these things.

@Justanothergrad you think it would cause psychic trauma to make them critique one of my own papers?

They hate it, but the data was really clear. I could stack people in order clearly based on their performance on that task - and it was consistent with everything else and the more prepared/serious clearly stood out (relative to research experience/CV, GRE, etc). The challenge was picking a paper that I didn't get them too lost on with the statistics. You could always give it to them about 2 days ahead of time and tell them to read it. At the end of the day, I also want people who can critique me / will critique me as students.

I was hesitant to have them design a study they were interested without some sort of guidance because (1) I don't expect they know the literature nearly well enough [I've read far too many undergrad proposals to ever think otherwise], (2) I don't expect those proposals to be realistic [see #1], (3) I want them to show they know my lab research, and (4) I want to find people who can be appropriately critical. I may swap to asking for them to generate the 'next step' based on the study with some bullet points of what I am looking for since they don't have a ton of time and would benefit from the scaffolding (e.g., what did the past study do well/where does it need improvement/what is your rationale for this new study/how would you design it/what do you expect to find and how is it different/greater than what we already know). That gives an opportunity for them to critique you, to develop their own stuff, and to show off what they know in a free form manner.

 

[Ollie123]

Mirror tracing task? Kidding. Sorta.

Its tough. I don't necessarily buy that <any> sort of interview is going to be effective for assessing ability above and beyond other measures, performance-based or not. That is especially true for something like what we do that isn't exactly production-based. To me, its just an effort to build rapport, make sure they are not completely devoid of social skills in a way that cannot be captured on paper and provide some vague notion of whether I could see myself getting along with them.

I'd just echo the advice above: A) Think hard about what specifically you want to get out of it; and B) Let them know in advance.

I'd perhaps steer you away from anything that comes with time pressure, like on-the-spot article critiques, etc. State anxiety is going to be higher during an interview even for the more chill folks. That will quite likely impair processing speed. Moreover, science is slow-moving. I worry something like an on-the-fly article critique would prioritize someone who can pick out 20 superficial flaws (They should have had 150 people instead of 100! They didn't have a bioverification of X!) over someone who sleeps on it and comes back the next day and says "They should never have done the study in the first place because their central theory is nonsense....and also here's a bunch of stuff they effed up." The former may be a good researcher, but won't be a good scientist and probably won't ever make it as a PI. Depends on what you want though. I don't think yours is one, but certainly some labs are "Crank out as many papers as you can using our plug & play formula" and for them the above approach may not make sense.