Without GREs, how do admissions committees screen?

[R. Matey]

---

Members don't see this ad.

And it’s not just taking the GRE once. The fact that there are those who have the means to fund themselves to take prep courses to improve scores (mine was $600 for an online on demand program) also perpetuates inequities.

This a weak argument. The GRE isn't driving most of the costs in graduate school admissions. Graduate school is expensive in general and the community library likely has resources for remedial study. Add in the cost of books (I used ETS and Princeton Review) and we're talking ~$100 for study materials.

The cost argument makes more sense when you consider the cost of applications in general. Since the prevailing wisdom is to apply to ~15 programs to maximize your chances, think 15 application fees, 15 transcript fees (if you only attended one institution), financing travel to interviews, costs of moving, etc.

 

[Big5Club]

This a weak argument. The GRE isn't driving most of the costs in graduate school admissions. Graduate school is expensive in general and the community library likely has resources for remedial study. Add in the cost of books (I used ETS and Princeton Review) and we're talking ~$100 for study materials.

The cost argument makes more sense when you consider the cost of applications in general. Since the prevailing wisdom is to apply to ~15 programs to maximize your chances, think 15 application fees, 15 transcript fees (if you only attended one institution), financing travel to interviews, costs of moving, etc.

This is all valid. But also doesn’t reflect my experience either. I’m a Latinx woman, and even though I had strong undergraduate quantitative training, spent a lot of money on study prep/GRE (~$1k total), and scored very well on the practice tests (163/162), I scored much lower on the actual test (159/158). I think the constant exposure to the idea that people like me do poorly on these exams engaged my stereotype threat. Just like the paper that Seven Costanza shared above suggests.

Simply ignoring the GRE won’t solve the systemic problem, as you stated. But I don’t think it’s not impactful. Ideally, we have to take a look at the problem and come together to make changes at each level of the system. But I don’t think that’s going to happen anytime soon, especially in this current climate.

 

[WisNeuro]

This a weak argument. The GRE isn't driving most of the costs in graduate school admissions. Graduate school is expensive in general and the community library likely has resources for remedial study. Add in the cost of books (I used ETS and Princeton Review) and we're talking ~$100 for study materials.

The cost argument makes more sense when you consider the cost of applications in general. Since the prevailing wisdom is to apply to ~15 programs to maximize your chances, think 15 application fees, 15 transcript fees (if you only attended one institution), financing travel to interviews, costs of moving, etc.

Exactly, the GRE is probably one of the smallest barriers to grad school acceptance for most. But, it is a convenient target for people to feel like they are doing something, even if that something doesn't really fix much of the problem at all.

 

[R. Matey]

This is all valid. But also doesn’t reflect my experience either. I’m a Latinx woman, and even though I had strong undergraduate quantitative training, spent a lot of money on study prep/GRE (~$1k total), and scored very well on the practice tests (163/162), I scored much lower on the actual test (159/158). I think the constant exposure to the idea that people like me do poorly on these exams engaged my stereotype threat. Just like the paper that Seven Costanza shared above suggests.

Simply ignoring the GRE won’t solve the systemic problem, as you stated. But I don’t think it’s not impactful. Ideally, we have to take a look at the problem and come together to make changes at each level of the system. But I don’t think that’s going to happen anytime soon, especially in this current climate.

1. Your scores are above the average for clinical psych overall though and my guess would be that your practice scores and actual scores are in the same confidence interval. Could be wrong though.
2. Don't disagree. I'm just saying kicking the GRE to the curb doesn't do much to solve the problem of underrepresentation in graduate school. The article I linked earlier provided an interesting middle path I thought.

 

[Big5Club]

1. Your scores are above the average for clinical psych overall though.
2. Don't disagree. I'm just saying kicking the GRE to the curb doesn't do much to solve the problem of underrepresentation in graduate school. The article I linked earlier provided an interesting middle path I thought.

But I spent $600 on testing materials to get there.

I guess I struggle with the “it’s not going to solve the problem” argument because the problem is not going to be solved anytime in our lifetime. Probably not in our children’s lifetime. We can’t even get the majority of people in this country to acknowledge that systemic oppression exists.

I’ll take a closer look at the methods in your paper when I get a break.

 

[WisNeuro]

I was curious, my local library system has 14 different GRE prep books available. Most of those with 5+ copies available. Why are people spending so much on GRE prep?

 

[Big5Club]

I was curious, my local library system has 14 different GRE prep books available. Most of those with 5+ copies available. Why are people spending so much on GRE prep?

I used the books for several weeks also, but my score wasn’t improving as much as I wanted and I was running out of practice exams to take on the weekends. I felt the on demand program with videos, explanations, and thousands of testing questions in the test bank that could be tailored to what I was struggling with was exponentially more helpful than the books were, and that is when my scores really started increasing.

 

[WisNeuro]

I used the books for several weeks also, but my score wasn’t improving as much as I wanted and I was running out of practice exams to take on the weekends. I felt the on demand program with videos, explanations, and thousands of testing questions in the test bank that could be tailored to what I was struggling with was exponentially more helpful than the books were, and that is when my scores really started increasing.

Choosing to take a more expensive option is different than saying everyone is forced to take the expensive option when other cheaper, and free options exist. Speaking of which, there are literally thousands of hours of free GRE prep videos on youtube if people want to go that way. I just don't buy the argument that people need expensive prep materials to do well on the GRE.

 

[Big5Club]

Choosing to take a more expensive option is different than saying everyone is forced to take the expensive option when other cheaper, and free options exist. Speaking of which, there are literally thousands of hours of free GRE prep videos on youtube if people want to go that way. I just don't buy the argument that people need expensive prep materials to do well on the GRE.

YouTube videos don’t provide a test bank of thousands of questions with individualized feedback and many of the other helpful perks that come along with the more expensive programs. And still, even low fee options are expensive for many. And as a white man whose trajectory is undoubtedly much more different than mine was, it makes sense that you don’t get the need for this level of preparation.

Cost and other access barriers disproportionately impact the underrepresented groups more. Stereotype threat impacts underrepresented groups more. Disability-related accommodations (or lack of) is cumbersome and very limited. And I don’t think the information the GRE gives us makes up for all of the above.

 

[WisNeuro]

YouTube videos don’t provide a test bank of thousands of questions with individualized feedback and many of the other helpful perks that come along with the more expensive programs. And still, even low fee options are expensive for many. And as a white man whose trajectory is undoubtedly much more different than mine was, it makes sense that you don’t get the need for this level of preparation.

Cost and other access barriers disproportionately impact the underrepresented groups more. Stereotype threat impacts underrepresented groups more. Disability-related accommodations (or lack of) is cumbersome and very limited. And I don’t think the information the GRE gives us makes up for all of the above.

I'm sure our trajectories were different, but growing up poor and paying my own way made me very cognizant of finances. At this point, you're arguing for extraordinary accommodations to make up for generally small effect sizes, even when comparable free and low cost options exist. Furthermore, we still haven't demonstrated that an actual problem with the GRE exists in clinicl psych admissions. But, if ad hominems make us feel better, I guess it's the trendy thing to do.

 

[Big5Club]

Choosing to take a more expensive option is different than saying everyone is forced to take the expensive option when other cheaper, and free options exist. Speaking of which, there are literally thousands of hours of free GRE prep videos on youtube if people want to go that way. I just don't buy the argument that people need expensive prep materials to do well on the GRE.

I'm sure our trajectories were different, but growing up poor and paying my own way made me very cognizant of finances. At this point, you're arguing for extraordinary accommodations to make up for generally small effect sizes, even when comparable free and low cost options exist. Furthermore, we still haven't demonstrated that an actual problem with the GRE exists in clinicl psych admissions. But, if ad hominems make us feel better, I guess it's the trendy thing to do.

Not an ad hominem attack and my apologies that it came off that way. It was more of a reflection on having to explain my experience and why I did what I did to people who don’t understand what it is like to take a non-traditional path, and deal with stereotype threat and microaggressions all along the way as a brown person. It’s the whole “have to work 50% harder than white people do” to end up in the same place.

And I wouldn’t consider dropping a standardized test that is only predictive of grad school success in the first year when evaluated alongside the GPA as an extraordinary accommodation.

I have to go complete my RQI requirement so I may not be able to respond for a while.

 

[WisNeuro]

Not an ad hominem attack and my apologies that it came off that way. It was more of a reflection on having to explain my experience and why I did what I did to people who don’t understand what it is like to take a non-traditional path, and deal with stereotype threat and microaggressions all along the way as a brown person. It’s the whole “have to work 50% harder than white people do” to end up in the same place.

And I wouldn’t consider dropping a standardized test that is only predictive of grad school success in the first year when evaluated alongside the GPA as an extraordinary accommodation.

I have to go complete my RQI requirement so I may not be able to respond for a while.

Assumptions about someone's trajectory are never very helpful as we don't know their family life growing up, their financial situations, how many family deaths they dealt with along the way, etc etc. And, I am not against dropping the GRE per se, I am against dropping it for made up reasons when we haven't even explored the issue all that well, and that we may actually be making the problem worse by doing so.

 

[Dazen]

Speaking as first generation college student from who grew up on food stamps, I actually did quite well on the GRE. In fact, my scores were factored into some scholarships I received during my time in graduate school, which further offset costs that weren't covered by my assistantship. IME, the greater barriers were access to adequate mentorship and quality data to write and publish on in addition to accurate career information. I think less people are scared off by the GRE than all of the other associated intangibles that come with attending graduate school such as relocating living on poverty wages (perhaps while supporting a family) for half a decade. Dropping the GRE is hardly worth congratulating anyone for removing systemic barriers in academia.

Wanted to pull back to a broader view of admissions barriers and highlight this post. From the little that I've seen so far, I would absolutely agree that lack of access to quality research mentorship and experience is one of the biggest barriers in graduate admissions. Multiple people in this thread have already said that lack of research productivity is a non-starter for admissions. Many students from marginalized backgrounds are unable to attend schools that would give these types of research opportunities, and some who are struggle to take advantage of these school's opportunities because of family commitments or having to take other jobs. I was fortunate that I could use research as my work study at the same or better pay rate, but I know this was a rare situation.

Separately, I'm curious to see whether schools continue to offer virtual interviews in addition to or in lieu of in-person. IMO this would significantly decrease the cost of the entire admissions process (flights, time off work, etc) and, based on anecdotes from last year, probably provides a similar view of candidates. Agree with @WisNeuro 's concern about making evidence-based decisions, so would love to see whether interview type leads to different outcomes.

 

[WisNeuro]

Separately, I'm curious to see whether schools continue to offer virtual interviews in addition to or in lieu of in-person. IMO this would significantly decrease the cost of the entire admissions process (flights, time off work, etc) and, based on anecdotes from last year, probably provides a similar view of candidates. Agree with @WisNeuro 's concern about making evidence-based decisions, so would love to see whether interview type leads to different outcomes.

I've talked about this one before, I actually wish we had funds available to pay for applicants travel rather than go all virtual. Along the way, physically seeing a site drastically changed my rankings, particularly at the internship and postdoc levels. IMO on-site interviews are hugely beneficial to the applicants, but don't change much about our rankings, as our rankings never changed much from application review in the first place.

 

[Mama Bear]

I have a question for admissions for folks. I was always told that, for PhD applicants to balanced scientist-practitioner programs, having a publication was rare and special. Having a pub basically made someone a shoe-in. Not having much exposure to the admissions side of things, I’m curious about the actual data. Can admissions folks estimate what % of all applicants have pubs when they apply?

 

[PsyStudent24]

Right. But for our program at least, proof of research experience/productivity is a non-negotiable. A perfect GRE for someone with little research experience is not going to make any cuts. And I believe research ability is more directly tied to the kind of work they will be doing in graduate school. We are able to glean more meaningful information from those experiences than someone’s score on the GRE.

And it’s not just taking the GRE once. The fact that there are those who have the means to fund themselves to take prep courses to improve scores (mine was $600 for an online on demand program) also perpetuates inequities.

As I mentioned before, investing in test prep material is unlikely to substantially boost an individual’s scores after a certain point. It’s also fairly common for competitive applicants to obtain stellar verbal scores but low quant scores, which will ultimately hurt their chances of gaining admission into a clinical psych phd program that is research focused. I do agree that accessibility is also a significant issue and some people simply cannot afford the outrageous test fees.

I wouldn’t be surprised if ETS suddenly decides to reduce test fees and increase accessibility, so that the GRE can continue to be used as a screener when reviewing applicants for admission.

In my graduate program, there was a cut-off score for GREs and applicants who didn’t score above it were automatically rejected before the faculty members could even review their materials. The admissions committee would set the bar higher or lower depending on the volume of applications. Other qualitative factors could offset a slightly lower score on one of the areas provided that it was but higher than the cut off and the other two scores were substantially higher.

 

[psych.meout]

I went to a clinical science program and was on the application review committee for 4 years. A low GRE would absolutely impact the review. The issue is that there are so many stellar applicants across the board that we’d need to find ways to filter people out.

I currently attend a clinical science program and have done this in my time in the program. How are you operationalizing low? The only way a person is not getting their other materials considered is with an exceptionally low GRE score. Moreover, if there are so many stellar applicants, it sounds like the GRE is being used as a tie breaker or to a way to filter when they have too many applicants to consider in depth. There's no way a program like Yale's can reasonably go through all the application materials for the over 500 applicants they had last year in just a month or so until they send out interview invites.

One year, I worked with one faculty member and we couldn’t narrow down her applicants beyond the top 17. She ended up having to do 17 Skype interviews to help her narrow it down to the top 3 invites. GRE definitely factored in. Someone also said that Quant score wasn’t that important. I would disagree. Maybe because my program was scientifically rigorous, it came into play a few times. An applicant with a Q score below 159 I’d say would be looked at with a much more critical eye.

Seriously?

I do think the accessibility issue is really being dismissed here. The cost absolutely impacts some people’s ability to take it (and potentially have to retake it). Additionally, there aren’t as many sites located near poorer areas, and it can be difficult to get there for many. Anybody who is dismissive about these concerns should acknowledge their privilege.

You can take the GRE from home now.

Schedule Your GRE General Test

The GRE General Test is offered 24 hours a day at home and most days of the week at a test center near you. Schedule your test and register for the GRE test today.

www.ets.org

Additionally, the GRE requires very specific identification information requirements that can pose barriers to undocumented student applicants.

For what proportion of applicants is this an issue?

Sure. I guess I was speaking to the larger issue that is being discussed here.

So each year in my program a grad student is assigned to a different faculty member to review applications. Two out of the 4 professors I worked with over the years filtered out lower GREs. They’d send me the spreadsheet with those folks already removed. The other 2 professors were more flexible and we would review the whole list. Everybody who applied to these professors would get at least a glance.

Sounds like the issue not the use of the GRE, but rather inconsistency in how it is used. Sounds like there should be some kind of empirically-based best practices developed....

Regarding low GREs and stellar applications. In large numbers? No. But there were a couple who I thought were excellent across the board, but scored in the mid-150s range. They were not invited. One also happened to be a POC.

How is an above average GRE score "low?"

(Updated) Average GRE Scores: Expert Analysis • PrepScholar GRE

Wondering how your GRE compares to other grad school applicants? Learn the official average GRE scores to see how you compare.

www.prepscholar.com

I've never seen anyone's application get tossed for an above average GRE score like that. Again, this is why it's important to operationalize your terms.

Again, I can’t say with any accuracy as 50% of the faculty I worked with filtered lower GREs out before I can see them. But as I mentioned, the 2 applicants I am thinking of that the other faculty considered had years of research experience, posters, and pubs. Stellar letters. One almost made the invite cut but was overruled by the rest of the faculty. This suggests to me that there are stellar applicants we miss when we filter out lower GREs. And if people are filtering out, which I suspect many programs/faculty do, how are we supposed to really gauge the percentage of folks who were stellar despite having lower GREs?

Sure, there are probably some great people who are missed because there are hard or soft filters (i.e., coming back around to those applications later if there are more interview slots or other interviewees didn't pan out), but the question is one of sensitivity and specificity. How many really competitive applicants are being missed when including the GRE vs. when it isn't included? This is an empirical question, not one that should be answered via anecdote and intuition.

On one hand, by keeping the GRE, a higher score can lift some disenfranchised applicants who had less exposure to opportunities due to life circumstances (I also doubt how impactful a high GRE is on applicants who have fewer other opportunities). On the other hand, by requiring it, you are excluding a sizable portion of people, likely lower SES and/or minority applicants, from even being considered. Either decision is likely going to disenfranchise some. For me, I feel uncomfortable with flat out excluding people with fewer resources.

People with more research experience are probably going be of greater means than those who don't have it. You need the money and time to go to a university with that level of research being done, which is cost prohibitive for many people, especially if they don't live in a large metro area or in a college town. Even if there are research opportunities at your institution, you need to be of enough means to be able to dedicate time and energy to attending lab meetings, doing research etc. People from lower SES backgrounds often have to work to put themselves through school and can't afford to forgo that time when they could be working. I have quite a few of these kinds of undergrads in the classes I teach. And sure, some people can get paid research jobs after graduation, but many others get unpaid volunteer research positions, especially as many of those paid positions require prior experience that they couldn't afford to get during undergrad. Again, people from lower SES backgrounds can't afford to do this, especially not for the amount of time per week and overall duration required to be competitive for grad school.

Conversely, the GRE is relatively cheap and quick by comparison.

With all this in mind, isn't it equally unfair to be filtering applicants based on the amount of research experience they have?

Right. But for our program at least, proof of research experience/productivity is a non-negotiable. A perfect GRE for someone with little research experience is not going to make any cuts. And I believe research ability is more directly tied to the kind of work they will be doing in graduate school. We are able to glean more meaningful information from those experiences than someone’s score on the GRE.

It's not really that simple. Just as there are differences in how faculty are using GRE scores in considering applicants, there are differences in how research experience is evaluated and in how productivity is obtained. Across disciplines, some faculty are quite stingy when it comes to authorship and sometimes don't even include people involved in the research who should be getting credit for their intellectual contributions, while others are too generous. They give authorship credit to undergrads who just did data entry and cleaning or ran participants, but justify it by those students being "bright" or having "potential" and wanting to give them a leg up into grad school where they're sure they'll succeed. Do you not think there are some troubling biases in the latter? What is the prevalence of each of these situations? I have no idea, just as you have no idea how often the GRE is causing truly competitive applicants to be dismissed outright.

And it’s not just taking the GRE once. The fact that there are those who have the means to fund themselves to take prep courses to improve scores (mine was $600 for an online on demand program) also perpetuates inequities.

This is all valid. But also doesn’t reflect my experience either. I’m a Latinx woman, and even though I had strong undergraduate quantitative training, spent a lot of money on study prep/GRE (~$1k total), and scored very well on the practice tests (163/162), I scored much lower on the actual test (159/158). I think the constant exposure to the idea that people like me do poorly on these exams engaged my stereotype threat. Just like the paper that Seven Costanza shared above suggests.

I mean, I'm white and I definitely could not have afforded to spend $1000 in study prep and GRE fees....

Simply ignoring the GRE won’t solve the systemic problem, as you stated. But I don’t think it’s not impactful. Ideally, we have to take a look at the problem and come together to make changes at each level of the system. But I don’t think that’s going to happen anytime soon, especially in this current climate.

What is "this current climate?"

I used the books for several weeks also, but my score wasn’t improving as much as I wanted and I was running out of practice exams to take on the weekends. I felt the on demand program with videos, explanations, and thousands of testing questions in the test bank that could be tailored to what I was struggling with was exponentially more helpful than the books were, and that is when my scores really started increasing.

Ok, but that sounds like a personal thing. Not every study method works for every person. I never could have afforded to spend hundreds of dollars on test prep, so I got free books online and from the library. I studied during my lunch break and on public transit on the way home (I couldn't afford a car either).

Wanted to pull back to a broader view of admissions barriers and highlight this post. From the little that I've seen so far, I would absolutely agree that lack of access to quality research mentorship and experience is one of the biggest barriers in graduate admissions. Multiple people in this thread have already said that lack of research productivity is a non-starter for admissions. Many students from marginalized backgrounds are unable to attend schools that would give these types of research opportunities, and some who are struggle to take advantage of these school's opportunities because of family commitments or having to take other jobs. I was fortunate that I could use research as my work study at the same or better pay rate, but I know this was a rare situation.

Separately, I'm curious to see whether schools continue to offer virtual interviews in addition to or in lieu of in-person. IMO this would significantly decrease the cost of the entire admissions process (flights, time off work, etc) and, based on anecdotes from last year, probably provides a similar view of candidates. Agree with @WisNeuro 's concern about making evidence-based decisions, so would love to see whether interview type leads to different outcomes.

Subjectively, it's not similar at all. Interviews last year provided maybe a couple of hours at best with the applicants for our lab. We tried lots of different things to encourage more contact, but it so awkward and artificial and we didn't want to pressure applicants into doing more than they felt comfortable or to cause them to be too fatigued for their real interviews. Pre-COVID we had more time at the dinner the night before interviews with our applicants than we did altogether with virtual interviews. I definitely would prefer to go back to in-person post-COVID and have the department to look for ways to pay for applicant travel.

As I mentioned before, investing in test prep material is unlikely to substantially boost an individual’s scores after a certain point.

Ok, but there seem to be two conflicting arguments being made here. Either the test prep material is relatively ineffective beyond a certain point, so it shouldn't matter that it's available, because there's no reasonable way for applicants who don't do well on high stakes testing to dramatically improve their performance, OR they do help, but they're expensive so it's just reinforcing existing SES disparities by allowing applicants of higher SES backgrounds to gain an advantage over meritorious applicants of lower SES backgrounds.

It’s also fairly common for competitive applicants to obtain stellar verbal scores but low quant scores, which will ultimately hurt their chances of gaining admission into a clinical psych phd program that is research focused.

Is it "fairly common?" How are you operationalizing that? This is another empirical question that you are answering with anecdote, belief, and intuition.

I do agree that accessibility is also a significant issue and some people simply cannot afford the outrageous test fees.

The test fees are one of the cheapest parts of applying for grad school. I spent more having my transcripts sent than for score reports and my application fees alone were cumulatively multiple times more expensive than what I paid for the GRE and all my score reports combined.

I'm not defending ETS, which sucks, but I'm sick of hearing this canard, especially as someone who had (and still has) no money.

I wouldn’t be surprised if ETS suddenly decides to reduce test fees and increase accessibility, so that the GRE can continue to be used as a screener when reviewing applicants for admission.

Maybe. As I said, they do suck.

In my graduate program, there was a cut-off score for GREs and applicants who didn’t score above it were automatically rejected before the faculty members could even review their materials. The admissions committee would set the bar higher or lower depending on the volume of applications. Other qualitative factors could offset a slightly lower score on one of the areas provided that it was but higher than the cut off and the other two scores were substantially higher.

Ok, what is your substitute to cut down the hundreds of applications to a manageable number that can be reasonably review in the 1-2 months before interview invites need to be extended?

 

[R. Matey]

In my graduate program, there was a cut-off score for GREs and applicants who didn’t score above it were automatically rejected before the faculty members could even review their materials. The admissions committee would set the bar higher or lower depending on the volume of applications. Other qualitative factors could offset a slightly lower score on one of the areas provided that it was but higher than the cut off and the other two scores were substantially higher.

The article I linked earlier specifically addresses this concern. Take a look.

 

[PsyStudent24]

This debate is getting heated.

 

[YoungFrancis]

The uncompensated labor that translates to research experience and products for applicants is more harmful than the GRE. Volunteering in a lab for multiple years is not feasible for many people and is far more financially burdensome than the cost of preparing for and taking the GRE.

Consider some basic undergraduate research experience: 10 hours/week for 2 years (let’s say 80 weeks). That’s $12k of lost income if you volunteer instead of get paid $15/hour.

Worse yet, plenty of people make a large time investments in volunteer research experience but are left with limited or no research products.

I don’t mind the GRE. I hate unpaid labor.

 

[NeuroPsychosis]

Will a good master's GPA be considered in the review of applicants? or adcoms will only look at undergraduate GPA?

 

[psych.meout]

Will a good master's GPA be considered in the review of applicants? or adcoms will only look at undergraduate GPA?

Yes, they look at master's GPA. One of the best (only?) reasons to do a terminal master's degree instead of just getting post-bacc research experience is to help compensate for a lower undergrad GPA.

 

[WisNeuro]

The SAT Isn’t What’s Unfair

MIT brings back a test that, despite its reputation, helps low-income students in an inequitable society.

www.theatlantic.com

Looked like a good addition to this discussion as it's a point raised before.

 

[ccool]

The SAT Isn’t What’s Unfair

MIT brings back a test that, despite its reputation, helps low-income students in an inequitable society.

www.theatlantic.com

Looked like a good addition to this discussion as it's a point raised before.

Except that SAT/ACT, undergrad admissions, and their predictive power for later outcomes are not equivalent to GRE, grad admissions, its predictive power for later outcomes.

GRE consistently predicts grad GPA, but i don’t know anyone who cares abt GPA in grad school.

 

[WisNeuro]

Except that SAT/ACT, undergrad admissions, and their predictive power for later outcomes are not equivalent to GRE, grad admissions, its predictive power for later outcomes.

GRE consistently predicts grad GPA, but i don’t know anyone who cares abt GPA in grad school.

I think you're missing the similarities here. There is an application process with different moving parts. The non-GRE moving parts that are especially important (e.g., research involvement, letters, essays/writing samples) are still situated to favor those with means, similar to this article. So, you essentially boot out one of the factors that is actually something of an equalizer, and emphasize parts of the process that are even more "inequitable."

 

[ccool]

I think you're missing the similarities here. There is an application process with different moving parts. The non-GRE moving parts that are especially important (e.g., research involvement, letters, essays/writing samples) are still situated to favor those with means, similar to this article. So, you essentially boot out one of the factors that is actually something of an equalizer, and emphasize parts of the process that are even more "inequitable."

I never said anything abt getting rid of the GRE.

I’m just speaking to its utility as a piece of data for its purported purpose.

 

[WisNeuro]

I never said anything abt getting rid of the GRE.

I’m just speaking to its utility as a piece of data for its purported purpose.

Sure, and that is a side issue in this specific instance. The more pressing issue is that by making a kneejerk decision in an effort to "solve" a problem with equity, there is a good chance that the problem was made worse.

 

[ccool]

Sure, and that is a side issue in this specific instance. The more pressing issue is that by making a kneejerk decision in an effort to "solve" a problem with equity, there is a good chance that the problem was made worse.

I disagree that it is a side issue. Even if we’re generally talking about the same matters, testing and undergraduate admissions is not fully comparable to GRE and graduate admissions. Thus the evidence supporting one does not fully speak to the issues related to the other.

I also disagree that all programs who decided to drop GRE requirement did so in a knee-jerk fashion. I am not certain that the problem was made worse.

Here’s a decent 2019 Science article that covers a number of matters and summarized recent research relevant to the movement to drop the GRE. Science | AAAS

 

[WisNeuro]

I disagree that it is a side issue. Even if we’re generally talking about the same matters, testing and undergraduate admissions is not fully comparable to GRE and graduate admissions. Thus the evidence supporting one does not fully speak to the issues related to the other.

I also disagree that all programs who decided to drop GRE requirement did so in a knee-jerk fashion. I am not certain that the problem was made worse.

Here’s a decent 2019 Science article that covers a number of matters and summarized recent research relevant to the movement to drop the GRE. Science | AAAS

Agree to disagree, then. But, the issue remains that the "fix" here is likely to have the opposite effect of its intended purpose due to a knee jerk reaction. The data here is infinitely more compelling than anything put out by the shortsighted drop the GRE crowd. They made a change without changing the more important aspects of the process, and likely disenfranchised certain people even more. They didn't bother to look at the whole process, but instead went for low hanging fruit that actually likely helped out those disenfranchised people.

 

[ccool]

Agree to disagree, then. But, the issue remains that the "fix" here is likely to have the opposite effect of its intended purpose due to a knee jerk reaction. The data here is infinitely more compelling than anything put out by the shortsighted drop the GRE crowd. They made a change without changing the more important aspects of the process, and likely disenfranchised certain people even more. They didn't bother to look at the whole process, but instead went for low hanging fruit that actually likely helped out those disenfranchised people.

Beyond MIT’s return to SAT requirement and stated reasoning in the article you linked, what evidence are you using to make the claim that dropping the GRE has *likely resulted in “disenfranchising certain people even more”?

Also, what is your evidence that “they made a change without changing the more important aspects of the process” and that “they didn’t bother to look at the whole process”?

 

[WisNeuro]

Beyond MIT’s return to SAT requirement and stated reasoning in the article you linked, what evidence are you using to make the claim that dropping the GRE has *likely resulted in “disenfranchising certain people even more”?

Also, what is your evidence that “they made a change without changing the more important aspects of the process” and that “they didn’t bother to look at the whole process”?

I said likely to disenfranchise certain individuals more, likely for the same exact reasons as stated in the article. As for not looking at the whole process, generally due to that being the only change made to the process on a systemic level, and based on what is likely intentionally misleading faulty data, a la the GRE equity articles that neglected to do an analysis on the largest confound despite having the data available.

 

[psycho1391]

People in this thread are talking about post-bacc research experience as if that is easy to get. Now, being on the other side of things and working in a research lab, we don't even interview folks without research experience during undergrad. I have brought up in research meetings as this disadvantages a lot of folks who can't sustain volunteer research during undergrad due to financial reasons. However, its fallen on deaf ears.

 

[ccool]

I said likely to disenfranchise certain individuals more, likely for the same exact reasons as stated in the article. As for not looking at the whole process, generally due to that being the only change made to the process on a systemic level, and based on what is likely intentionally misleading faulty data, a la the GRE equity articles that neglected to do an analysis on the largest confound despite having the data available.

Hmm. Ok. These are just strong claims about keeping/dropping the GRE that I have not seen realized in my direct or indirect observations or experiences, nor in any research I have seen. I also have not seen any data to support the argument for grad admission that has been applied to undergraduate admission.

We can talk about other inequities in the grad admission process without making unfounded claims that the GRE is necessary to right other inequities or that institutional decisions regarding using the GRE or not in grad admissions were the only changes that were made and that data that may or may not have been used was likely intentionally misleading or faulty.

 

[WisNeuro]

We can make whatever claims we want to, apparently, without any solid data. That's exactly what the "drop the GRE" crowd has indeed done. As the article outlines we have seen what happens when we do a very similar thing at the undergrad admissions level, and it's very easy to see the similarities at the grad level given what goes into the selection process. Just sucks that students will bear the brunt of rash decision making. Doesn't personally affect me at this point, I have the luxury at just shaking my head at the stupid decisions that get made in the field.

 

[ccool]

We can make whatever claims we want to, apparently, without any solid data. That's exactly what the "drop the GRE" crowd has indeed done. As the article outlines we have seen what happens when we do a very similar thing at the undergrad admissions level, and it's very easy to see the similarities at the grad level given what goes into the selection process. Just sucks that students will bear the brunt of rash decision making. Doesn't personally affect me at this point, I have the luxury at just shaking my head at the stupid decisions that get made in the field.

Whereas the SAT/ACT predicts meaningful outcomes at the undergraduate level (and beyond.. eg see relationship between undergraduate GPA and professional school admission) the GRE does not predict meaningful outcomes for graduate school. Thus, despite and because of the other inequities, including those related to the test itself, we can follow how the SAT/ACT serves as a useful and in some cases critical data point to help guide admissions decisions at the undergraduate level.

Despite and because of the other inequities in grad admissions process including those related to the test itself, and given that the GRE is not a meaningful predictor of grad school (or beyond) success, without using intentional, evidence-based holistic review processes, the addition of the GRE in grad school admissions ultimately serves as a barrier to equitable admissions processes.

Even if there is data at the graduate level saying that in some cases, URM and 1st gen applicants with low to average research experience are being considered bc of their high GRE score (I have not seen this), that does not resolve the more common situation of URM and 1st gen applicants with low to average GRE scores and low or average or stellar research experience etc. being filtered out earlier in the process. Of course all people with the low to average score would get filtered out, but URM and 1st gen applicants would be/are disproportionately affected bc of the documented score inequities. Furthermore, doing either, having a grad interview or admission decision be ultimately based on (eg otherwise similar candidates on paper via LORs, research, undergrad gpa but one is 50th percentile and the other 90th) someone’s low or average or high GRE score is absurd given that it does not predict meaningful metrics of success.

With the constant that is societal inequality, there doesn’t seem to be any easy heuristics to apply here. If ppl are looking for a quantitative metric to ultimately differentiate applicants, using one that has actual predictive power for its purported purpose would be a start.

In the meantime, I’ll be interested to see the data that emerges over the next few years regarding outcomes associated with some of these recent shifts.

 

[psych.meout]

Whereas the SAT/ACT predicts meaningful outcomes at the undergraduate level (and beyond.. eg see relationship between undergraduate GPA and professional school admission) the GRE does not predict meaningful outcomes for graduate school. Thus, despite and because of the other inequities, including those related to the test itself, we can follow how the SAT/ACT serves as a useful and in some cases critical data point to help guide admissions decisions at the undergraduate level.

https://journals.sagepub.com/doi/pdf/10.1177/0013164409344508?casa_token=WUnYKH2t1EgAAAAA:K5VD0SlDiAja3YLtDw0ruPrBR5XYND6xGHzLelymbWQKCfzwOCTwnyxqnUGlhAmOYMyFXZ8myA

Despite and because of the other inequities in grad admissions process including those related to the test itself, and given that the GRE is not a meaningful predictor of grad school (or beyond) success, without using intentional, evidence-based holistic review processes, the addition of the GRE in grad school admissions ultimately serves as a barrier to equitable admissions processes.

It's not an either-or situation. Programs can and do use an evidence-based holistic review process that includes the GRE.

Even if there is data at the graduate level saying that in some cases, URM and 1st gen applicants with low to average research experience are being considered bc of their high GRE score (I have not seen this), that does not resolve the more common situation of URM and 1st gen applicants with low to average GRE scores and low or average or stellar research experience etc. being filtered out earlier in the process. Of course all people with the low to average score would get filtered out, but URM and 1st gen applicants would be/are disproportionately affected bc of the documented score inequities. Furthermore, doing either, having a grad interview or admission decision be ultimately based on (eg otherwise similar candidates on paper via LORs, research, undergrad gpa but one is 50th percentile and the other 90th) someone’s low or average or high GRE score is absurd given that it does not predict meaningful metrics of success.

Ok, what do you propose as tie-breakers to determine admissions if applicants are equal on every other metric?

With the constant that is societal inequality, there doesn’t seem to be any easy heuristics to apply here. If ppl are looking for a quantitative metric to ultimately differentiate applicants, using one that has actual predictive power for its purported purpose would be a start.

In the meantime, I’ll be interested to see the data that emerges over the next few years regarding outcomes associated with some of these recent shifts.

But again, you're not addressing the point being made, which agrees with the premise that there is inequality, but notes that getting research experience and other more important metrics are even more biased against these same groups than getting higher GRE scores. It's much, much cheaper, easier, and more likely for someone to get a higher GRE score than them being able to find research positions, move for them if they aren't available in their area, support themselves (and their family) if the positions are unpaid, etc. And none of this gets into the disparities in those research positions that likely mirror those from other work environments (e.g., being passed over for opportunities).

 

[ccool]

https://journals.sagepub.com/doi/pdf/10.1177/0013164409344508?casa_token=WUnYKH2t1EgAAAAA:K5VD0SlDiAja3YLtDw0ruPrBR5XYND6xGHzLelymbWQKCfzwOCTwnyxqnUGlhAmOYMyFXZ8myA

This article you cite is from 2009. Im curious abt any recent changes given the more recent push for increasing diversity in the pipeline. Do these same associations hold? Additionally it reiterates that GRE predicts grad GPA. I still maintain that no one cares abt grad GPA. Here’s an article (using data from 2001-2011) showing potential impact of filtering out students by GRE quant score. APA PsycNet

Regardless of GRE score, students are either meeting the required metrics or not. If they are not, they are being put on plans to support their ability to do so or are being counseled out/leaving.

It's not an either-or situation. Programs can and do use an evidence-based holistic review process that includes the GRE.

Yes, this is what I advocate for, regardless whether they use the GRE at all. But how holistic is a review if admissions decisions still come down to GRE scores? Also, I am not entirely sure how various institutions operationalization and enact holistic review beyond saying that they do. Here’s a nice experimental study using a simple and efficacious instruction intervention to encourage faculty to review candidates holistically.

Frontiers | Instructional Set Moderates the Effect of GRE on Faculty Appraisals of Applicant Competence: A Vignette Study With Implications for Holistic Review

ment to create more equitable and holistic admission review processes, faculty continue to place strong emphasis on a single piece of information when making...

www.frontiersin.org

Ok, what do you propose as tie-breakers to determine admissions if applicants are equal on every other metric?

I don’t know, probably “fit,” or some other metric that is actually predictive of success. And of course we can argue that fit is a biased and subjective index, but I maintain that everything that humans engage with and manipulate is on some level. I’ve seen some older data that suggests that the psych GRE is predictive of metrics including research output and completion. Or perhaps some kind of performance task that assesses (capacity for) critical thinking, problem solving, decision making, flexibility, humility and other impt characteristics that make for a successful clinical psychologist. I don’t know.

But again, you're not addressing the point being made, which agrees with the premise that there is inequality, but notes that getting research experience and other more important metrics are even more biased against these same groups than getting higher GRE scores. It's much, much cheaper, easier, and more likely for someone to get a higher GRE score than them being able to find research positions, move for them if they aren't available in their area, support themselves (and their family) if the positions are unpaid, etc. And none of this gets into the disparities in those research positions that likely mirror those from other work environments (e.g., being passed over for opportunities).

I’m not arguing this point. Noting other potential biases in admissions processes is related but not interchangeable to the GRE is necessary point. The current argument is that the GRE is necessary in order to mitigate other biases inculcated in grad admissions processes. And furthermore that the GRE/use of the GRE is less biased and exclusionary than these other potential sources of inequity. I disagree with the former for all the reasons I stated. I don’t think there is data that speaks to the latter. I would argue that it is differently biased, but I am not sure of a hierarchy among them.

 

[WisNeuro]

This article you cite is from 2009. Im curious abt any recent changes given the more recent push for increasing diversity in the pipeline. Do these same associations hold? Additionally it reiterates that GRE predicts grad GPA. I still maintain that no one cares abt grad GPA. Here’s an article (using data from 2001-2011) showing potential impact of filtering out students by GRE quant score. APA PsycNet

Regardless of GRE score, students are either meeting the required metrics or not. If they are not, they are being put on plans to support their ability to do so or are being counseled out/leaving.


Yes, this is what I advocate for, regardless whether they use the GRE at all. But how holistic is a review if admissions decisions still come down to GRE scores? Also, I am not entirely sure how various institutions operationalization and enact holistic review beyond saying that they do. Here’s a nice experimental study using a simple and efficacious instruction intervention to encourage faculty to review candidates holistically.

Frontiers | Instructional Set Moderates the Effect of GRE on Faculty Appraisals of Applicant Competence: A Vignette Study With Implications for Holistic Review

ment to create more equitable and holistic admission review processes, faculty continue to place strong emphasis on a single piece of information when making...

www.frontiersin.org

I don’t know, probably “fit,” or some other metric that is actually predictive of success. And of course we can argue that fit is a biased and subjective index, but I maintain that everything that humans engage with and manipulate is on some level. I’ve seen some older data that suggests that the psych GRE is predictive of metrics including research output and completion. Or perhaps some kind of performance task that assesses (capacity for) critical thinking, problem solving, decision making, flexibility, humility and other impt characteristics that make for a successful clinical psychologist. I don’t know.

I’m not arguing this point. Noting other potential biases in admissions processes is related but not interchangeable to the GRE is necessary point. The current argument is that the GRE is necessary in order to mitigate other biases inculcated in grad admissions processes. And furthermore that the GRE/use of the GRE is less biased and exclusionary than these other potential sources of inequity. I disagree with the former for all the reasons I stated. I don’t think there is data that speaks to the latter. I would argue that it is differently biased, but I am not sure of a hierarchy among them.

This is not the argument that has been proposed by myself, or several others who are critical of the "drop the GRE crowd."

 

[ccool]

This is not the argument that has been proposed by myself, or several others who are critical of the "drop the GRE crowd."

I think you're missing the similarities here. There is an application process with different moving parts. The non-GRE moving parts that are especially important (e.g., research involvement, letters, essays/writing samples) are still situated to favor those with means, similar to this article. So, you essentially boot out one of the factors that is actually something of an equalizer, and emphasize parts of the process that are even more "inequitable."

Sure, and that is a side issue in this specific instance. The more pressing issue is that by making a kneejerk decision in an effort to "solve" a problem with equity, there is a good chance that the problem was made worse.

Agree to disagree, then. But, the issue remains that the "fix" here is likely to have the opposite effect of its intended purpose due to a knee jerk reaction. The data here is infinitely more compelling than anything put out by the shortsighted drop the GRE crowd. They made a change without changing the more important aspects of the process, and likely disenfranchised certain people even more. They didn't bother to look at the whole process, but instead went for low hanging fruit that actually likely helped out those disenfranchised people.

 

[WisNeuro]

Yes, and no one has said that the GRE is necessary. You can be as obtuse about that as you wish to fit your narrative. The argument is that it is currently a part of a multifaceted admissions process. And, that in attempting to lead a crusade of some sort, the soft powers that be wanted the GRE removed, generally using flawed data to support their views. And, that they did so without thinking through the process, and, in turn, may be hurting those that they claimed to want to help. So, they argument is not that the GRE is necessary, it's that it was removed without thought and not replaced with anything that was better. In fact, it likely just puts more emphasis on parts of the application that are even more out of reach with disadvantaged individuals.

 

[psych.meout]

This article you cite is from 2009. Im curious abt any recent changes given the more recent push for increasing diversity in the pipeline. Do these same associations hold? Additionally it reiterates that GRE predicts grad GPA. I still maintain that no one cares abt grad GPA.

Except the article is not just about GPA, but also faculty ratings of students across a variety of relevant metrics of student performance across domains.

Here’s an article (using data from 2001-2011) showing potential impact of filtering out students by GRE quant score. APA PsycNet

That article isn't really matching up with what anyone here is advocating.

The field of psychology must racially/ethnically diversify to create a workforce that can meet the needs of education, training, and interventions in an increasingly pluralistic society. Systemic bias in psychology doctoral programs’ admissions process may partially account for relatively few psychologists being underrepresented minorities (URMs). The use of the Graduate Record Examination Quantitative score (GRE-Q) is one important modifiable barrier. The purpose of the current study is to go beyond replicating the association between the GRE-Q and desired doctoral outcomes by examining if a cut-off score for the GRE-Q as a proxy for potential to succeed in psychology doctoral programs disproportionately impacts URMs. Participants (N = 226) were psychology doctoral students at a Carnegie-classified Highest Research Activity (R1) large Midwestern university, who were admitted to graduate school from 2001 to 2011. Our findings show that, while controlling for undergraduate grade point average (GPA) and prior master’s degree attainment, the GRE-Q predicted grades in two required graduate statistics courses and overall graduate GPA. Importantly, all students, regardless of their GRE-Q score, demonstrated competence in their statistics coursework, as assessed by their course grades. Moreover, we found that guidelines that bar admission into the psychology doctoral program for students with low GRE-Q scores would have disproportionately impacted URMs, resulting in 44% being barred admission versus only 17% of their White/Asian/Pacific Islander counterparts. Practical implications include introducing holistic review protocols into the admissions process, while educating faculty on how heavy emphasis on the GRE-Q contributes to inequitable exclusion of capable URMs.

I don't and I haven't seen anyone else here endorse barring students from admission solely on the basis of GRE scores. Instead, it seems that everyone is arguing on behalf of a holistic evaluation that includes the GRE in some way.

Regardless of GRE score, students are either meeting the required metrics or not. If they are not, they are being put on plans to support their ability to do so or are being counseled out/leaving.

You're missing the point, which is that programs are looking for ways to avoid this kind of attrition, not just have policies for remediation plans and expulsion. Spots in doctoral programs are an extremely scarce resource, so programs want to avoid attrition as much as possible, which also minimizes sunk costs. Moreover, they want to reduce the opportunity cost of missing out on students who would be able to complete their programs when the matriculated peers ended up failing out.

Yes, this is what I advocate for, regardless whether they use the GRE at all. But how holistic is a review if admissions decisions still come down to GRE scores? Also, I am not entirely sure how various institutions operationalization and enact holistic review beyond saying that they do. Here’s a nice experimental study using a simple and efficacious instruction intervention to encourage faculty to review candidates holistically.

Frontiers | Instructional Set Moderates the Effect of GRE on Faculty Appraisals of Applicant Competence: A Vignette Study With Implications for Holistic Review

ment to create more equitable and holistic admission review processes, faculty continue to place strong emphasis on a single piece of information when making...

www.frontiersin.org

Again, do you not see how this article is supporting the point we've been making that the entire process is biased, but the focus on GRE scores is to the neglect of addressing the pervasive issues across criteria?

While there is movement to create more equitable and holistic admission review processes, faculty continue to place strong emphasis on a single piece of information when making admissions decisions: standardized test scores. This study used an experimental design to test whether instructions provided to faculty prior to assessing doctoral applicants could support holistic review by reducing the weight of the general record examination (GRE) in faculty appraisals of competence and merit for graduate study. Tenured and/or tenure-track faculty (N=271) were randomly assigned to one of three instructional conditions: Control (no instruction), “Diamond in the Rough,” and “Weed Out.” In addition, faculty participants were randomly assigned to read one of two vignettes of a prospective first-generation student who either received high or average GRE scores. Faculty then rated the applicant’s competence using a three-item survey. As expected, faculty who read the vignette describing the candidate with the high GRE rated him as more competent than faculty who read the average GRE vignette. In addition, being instructed to seek out diamonds in the rough buffered the effect of the GRE score on competence. Faculty were also asked to indicate whether they would need additional information to make an admissions decision. They were more likely to ask about grades and research skills than about psychosocial factors that might contextualize the candidate’s performance and perceived competence. The results of this study have implications for creating more equitable doctoral admissions processes that center equity, diversity, and inclusion in decision making.

If you are going to complain about the psychosocial factors affecting GRE performance and score interpretation, you have to be honest and consistent about applying this same scrutiny to other factors, and this study is indicating that this really isn't occurring.

I don’t know, probably “fit,” or some other metric that is actually predictive of success. And of course we can argue that fit is a biased and subjective index, but I maintain that everything that humans engage with and manipulate is on some level. I’ve seen some older data that suggests that the psych GRE is predictive of metrics including research output and completion. Or perhaps some kind of performance task that assesses (capacity for) critical thinking, problem solving, decision making, flexibility, humility and other impt characteristics that make for a successful clinical psychologist. I don’t know.

1. Again, the question was what to use when all other metrics, including fit, are equivalent. If you don't have an answer, then why isn't the GRE sufficient for the time being? This is exactly what @WisNeuro and others are saying here. You want to drop the GRE but without first developing some kind of replacement (hopefully a better one) and without addressing or even by minimizing other sources of bias compared to the GRE because you have an a priori bias about the GRE.
2. Your point about fit is a perfect example of you minimizing or reducing the impact of sources of bias that you like while inflating that of sources of bias you don't like, e.g., GRE. You keep mischaracterizing what other people are arguing, which is not a full-throated defense of the GRE, but rather a criticism of the rigor by which bias is being evaluated and addressed. That you are defending certain criteria based from being biased on the argument that bias pervades every activity, but aren't applying this same defense to the GRE is just obtuse and hypocritical.

I’m not arguing this point. Noting other potential biases in admissions processes is related but not interchangeable to the GRE is necessary point. The current argument is that the GRE is necessary in order to mitigate other biases inculcated in grad admissions processes. And furthermore that the GRE/use of the GRE is less biased and exclusionary than these other potential sources of inequity. I disagree with the former for all the reasons I stated. I don’t think there is data that speaks to the latter. I would argue that it is differently biased, but I am not sure of a hierarchy among them.

No one said that the GRE is "necessary," we're saying that the objections to the GRE are just as, if not more, prevalent in the other criteria in which applicants are judged and people trying to strip away the GRE immediately are rigorously evaluating the bias or potential impact in getting rid of the GRE and keeping the other criteria. We're saying that the same arguments used against the GRE can be leveled against the other criteria, but there is not the same movement to get rid of them, which is obtuse.

And are you really disputing that it's easier to just practice and take a test than to uproot one's life for research positions, being able to survive unpaid or underpaid while doing so, and dealing with all the well-researched biases that come with employment?

 

[ccool]

Yes, and no one has said that the GRE is necessary. You can be as obtuse about that as you wish to fit your narrative. The argument is that it is currently a part of a multifaceted admissions process. And, that in attempting to lead a crusade of some sort, the soft powers that be wanted the GRE removed, generally using flawed data to support their views. And, that they did so without thinking through the process, and, in turn, may be hurting those that they claimed to want to help. So, they argument is not that the GRE is necessary, it's that it was removed without thought and not replaced with anything that was better. In fact, it likely just puts more emphasis on parts of the application that are even more out of reach with disadvantaged individuals.

Hmm. I am consistently struck by your baseless assumptions about other people’s processes and intentions.

I do not understand how your argument does not rest on the premise that given everything else removing it is bad because it is needed? If it was not needed it would not need to be replaced and it would not need to remain as a potential equalizer? What am I missing?

If you’re arguing that a removed GRE needs to be replaced with something else because what remains is still subject to bias, ok… except all subject to bias…I guess we could move to a full lottery with zero criteria besides applying. 😉

Except the article is not just about GPA, but also faculty ratings of students across a variety of relevant metrics of student performance across domains.

And what are faculty ratings associated with?

That article isn't really matching up with what anyone here is advocating.

I don't and I haven't seen anyone else here endorse barring students from admission solely on the basis of GRE scores. Instead, it seems that everyone is arguing on behalf of a holistic evaluation that includes the GRE in some way.

This is what happens in practice, however. When you say, if students are equal on all other metrics except the GRE, without it, how do you decide, you are presumably saying that the student with the higher GRE score should be chosen. So then to your point below….

You're missing the point, which is that programs are looking for ways to avoid this kind of attrition, not just have policies for remediation plans and expulsion. Spots in doctoral programs are an extremely scarce resource, so programs want to avoid attrition as much as possible, which also minimizes sunk costs. Moreover, they want to reduce the opportunity cost of missing out on students who would be able to complete their programs when the matriculated peers ended up failing out.

If programs are seeking to avoid attrition and increase enrollment of students who have higher likelihood of success, based on the current data, use of the GRE is not effective to meet that goal.

Again, do you not see how this article is supporting the point we've been making that the entire process is biased, but the focus on GRE scores is to the neglect of addressing the pervasive issues across criteria?



If you are going to complain about the psychosocial factors affecting GRE performance and score interpretation, you have to be honest and consistent about applying this same scrutiny to other factors, and this study is indicating that this really isn't occurring.

Yes, exactly, I agree.

1. Again, the question was what to use when all other metrics, including fit, are equivalent. If you don't have an answer, then why isn't the GRE sufficient for the time being? This is exactly what @WisNeuro and others are saying here. You want to drop the GRE but without first developing some kind of replacement (hopefully a better one) and without addressing or even by minimizing other sources of bias compared to the GRE because you have an a priori bias about the GRE.

The GRE isn’t sufficient for the time being because it isn’t effective for its purported purposes. The other metrics that are also subject to bias and that can present inequitable barriers to admission are useful for their intended purposes. So there is a sound argument to keep them in addition to working to make them more equitable. That does not hold for the GRE. Basically, even if we were to make the GRE the most bias-proof test by ensuring equitable access to resources, opportunities, and quality K-12 education across people, it would still not matter if the test was not predictive of meaningful indicators of success/did not distinguish between applicants who would and would not succeed in grad school or in their later career. And in that case, what is the point of its use?

2. Your point about fit is a perfect example of you minimizing or reducing the impact of sources of bias that you like while inflating that of sources of bias you don't like, e.g., GRE. You keep mischaracterizing what other people are arguing, which is not a full-throated defense of the GRE, but rather a criticism of the rigor by which bias is being evaluated and addressed. That you are defending certain criteria based from being biased on the argument that bias pervades every activity, but aren't applying this same defense to the GRE is just obtuse and hypocritical.

My point hinges on the fact that the GRE is not useful. It creates barriers, exacerbates inequities, reiterates bias, and is not useful.

No one said that the GRE is "necessary," we're saying that the objections to the GRE are just as, if not more, prevalent in the other criteria in which applicants are judged and people trying to strip away the GRE immediately are rigorously evaluating the bias or potential impact in getting rid of the GRE and keeping the other criteria. We're saying that the same arguments used against the GRE can be leveled against the other criteria, but there is not the same movement to get rid of them, which is obtuse.

Ah, helpful. So you are saying that your argument is not in fact that keeping the GRE as a metric for admissions is important and that getting rid of it is a bad idea because of X reasons, including and not limited to being a useful metric on which to differentiate applicants and reducing the impact of other biased processes. Rather, it is that if we are going to be critical of the GRE and take steps to remove it as a requirement, then we should be critical of other hard and soft requirements? And if we are not advocating for the removal of other requirements known to inject bias into the process, then it is hypocritical to do so for the GRE?

If that is what you are saying, then yes, I agree that it is important to remain critical of all of the pieces and to continue to research predictors and outcomes associated with each. I also think it is important to develop and implement interventions to increase equity and assess outcomes related to those interventions. For example, the steady increase in federal and NGO funding to support mentorship and training programs designed to support URM and first gen students’ entrance and success in graduate programs and professional careers is really exciting. These interventions were based on data that indicated that one barrier to grad admission and success for these students was lack of adequate resources, and lack of access to opportunities, networks, and unwritten knowledge. I imagine that there is ample research suggesting these interventions have been at least somewhat effective in recruitment and retention efforts of URM students. I see universities’ experimentation with removing the GRE as another intervention based on data indicating that the scores are not predictive of anything useful and that it’s use disproportionately affects students they are actively trying to recruit. Could I offer criticisms of their methodology? Sure. But at this point, I’m looking forward to seeing outcomes. And based on current data, I expect that these changes will increase the number of generally qualified students, particularly URM students, in grad school and subsequently the field, something I think will only serve students, psychology, and the public. I’m not sure how effective or widespread other current interventions (e.g. fee waivers, free study materials) related to reducing inequities associated with the GRE are.

So to the second point, again, I disagree because of the differences in the utility of the GRE vs other factors in predicting success.

And are you really disputing that it's easier to just practice and take a test than to uproot one's life for research positions, being able to survive unpaid or underpaid while doing so, and dealing with all the well-researched biases that come with employment?

This is a laden question and beside the point…to me.

We don’t have to agree. And we’ll all see what is borne from these changes in several years. :shrugs:

 

[WisNeuro]

Hmm. I am consistently struck by your baseless assumptions about other people’s processes and intentions.

I do not understand how your argument does not rest on the premise that given everything else removing it is bad because it is needed? If it was not needed it would not need to be replaced and it would not need to remain as a potential equalizer? What am I missing?

If you’re arguing that a removed GRE needs to be replaced with something else because what remains is still subject to bias, ok… except all subject to bias…I guess we could move to a full lottery with zero criteria besides applying. 😉

Here we agree.

 

[ccool]

Here we agree.

Lol k.

 

[R. Matey]

Related, but I really liked this piece by McWhorter. I'll own my bias as a McWhorter fan though.

Opinion | Making the SAT and ACT Optional Is the Soft Bigotry of Low Expectations (Published 2022)

Intended or not, it sends a message that some students of color are less capable.

www.nytimes.com