APS and Potentially Biased Editorial Experience

[borne_before]

Correct me if I'm wrong, but I don't believe the critiques were ever published.

Good point - I'll adjust that above.

---

Members don't see this ad.

 

[ClinicalABA]

Here is Roberts et al. (2020) paper that started this whole process:

https://journals.sagepub.com/doi/pdf/10.1177/1745691620927709?utm_source=miragenews&utm_medium=miragenews&utm_campaign=news

I encourage you to read it yourself, but I know that is not possible for everyone. I will summarize it (rather that doing my actual work, which of course I'll catch up on this weekend). Sorry to turn this into an online "journal club," but i like the topic and your participation in optional!

Pretty straightforward structured literature review, with all the ususual strengths and limitations (acknowledged by the authors) typical found with such things. They do a particularly good job, IMHO, of laying out their procedures, including how they idientified the race of the authors/editors (as an aside, sounds like an extremely grueling process, going through 20,000+ articles!). There findings are presented clearly. Here's my summary:
-The authors present some previous research eveidence suggesting that there are differences in between races on some basic psychological (including cognitive) processes.
-Studies specifically focused on race/diversity issues are relatively uncommon in the past 5 decades. When they are published, authors are more likely to be people of color (POC)
-Over the past few decades there has been and increase in articles focused on race/diversity in social psychology and, particularly, developmental psychology journals (avg. about 10% or articles in the 2010 with this focus. There has been no noticeable change in this are in cognitive psychology journals (around 1% in the 2010s).
-Editors in Chief are predominately white and, in the case of cognitive psych journals, pretty much exclusively white
-Journal with white editors and boards differentially publish more research by white authors done with white subjects than Journals with POC in editorial positions
-The effects are statistically significant.

That's the objective stuff. The authors then go on to make some recommendations for Journals in addressing the issue (which they clearly see as a problem). My summary:
-Journals should include a statement about the extent of their commitment to diversity (e.g., on the jorunal web page)
-The composition of the editorial and review boards (in general, not just for "special issues") should reflect the racial diversity in line with the community as a whole (national level) or within psychology
-Journals should identify whether or not the sample contained primarily White, Educated, Industrialized, Rich, and Democratic samples. (Roberts acronymizes this as WEIRD. While I am from this population, admittedly "rich" only by comparative standards to the rest of the world, I find this type of "wink and a nod" deprecation humorous, non-offensive, and- frankly, a bit overdue!). They point out that journal already vet and identify samples that come from preregistered studies or publicly available data sets.
-Journals should release public diversity reports annually
-Journals should establisjh a diversity task force to monitor progress in these areas.
Note that they really only reviewed the most influential journals in each area.

They also make recommendations to authors. My summary:
-Detail the racial demographics of samples
-Justify the racial demographics of samples
-Include constraints on generality statements
-Include positionality statements (i.e., how the identities of the authors relate to the content of the study)

Overall, my is that the objective findings are what they are. I see no major methodological flaws or inconsistencies, and the limitations of the sample are clearly acknowledged by the authors. My view of their recommondations (for point of reference I am a middle-aged, mid-to-upper middle class, hetero, cis-gendered, married, graduate educated white male who learns far left politically on social issues and more "standard" left on financial issues) is that most are easily implemented by spending a few minutes adding some additional text to publications. Some (e.g., changes to editorial boards, require much more effort, but still follow directly from the findings of the current research. In no place (other than potentially where they use the WEIRD acronym!), did I find it to be overly "activist", unreasonable pushy regarding an ideology, or suggesting changes to the overall scientific method (but rather to how the results of the method are categorized and disseminated). They clearly stated that they were not directing their subjective findings towards scientists, but towards the science.

Not having seen the full responses from the 3 other authors in the "debate," it's hard to conclusively form an opinion. If we assume that Roberts "rebuttal" article contained a factual account, seems like some cognitive psychologist may have taken it all a little personally and immediately reacted defensivley based on their own ideology. I certainly hope to have access to their papers.

 

[Ollie123]

Appreciate the summary. The original paper does sound very reasonable. I do question the global need for positionality statements. It makes sense for some topics, but I'm 100% certain we will simply mandate token statements that don't mean anything and require them even in bizarre irrelevant scenarios (i.e., this may make sense for exchanges like the above, less so for a study examining the effects of prazosin on swim behavior in zebrafish). This also has a chance to spiral out of control quickly as many identities beyond race matter. If we can all collectively agree to use common sense in where/when we implement this rule I'm perfectly fine with it, but our ability to use common sense is sorely lacking.

All that said, I'm siding more with Roberts after seeing this. A lot really depends on the other critiques though.

FYI - Roberts didn't invent the "WEIRD" acronym. That's been around for 10+ years and is pretty widely used. Its a real problem in psychology, particularly the areas that rely heavily on undergrad samples (e.g., social psych).

I do find some of the recommendations baffling, but it may just be the type of work I do and where I publish. I don't think I've ever <not> included a sample characteristics table. I very, very, very rarely see anyone submit a paper without reporting sample characteristics and I've been reviewer/editor on plenty of papers that were absolute garbage. I can't remember the last time I saw an all-white sample or even a nearly all-white sample. Its usually generally representative of the region, with expected variation if a small study. The point on generalization is well-taken though. Even if a study matches US census bureau demographic distributions, you can't reasonably assume it generalizes to individual subsets of the US population because you aren't powered to detect effects in the smaller groups. This is why epi studies will often oversample minority groups and then down-weight them for prevalence estimates - not doing so pretty much precludes viable subsample analyses. Representativeness is of course still an issue, but race is usually pretty far down the list of reasons why.

 

[Shiori]

I have seen some cognitive psych papers not report demographics. One of my old professors justified it by saying they weren't necessary because the work was just focused on brains and not culture.

 

[Ollie123]

I have seen some cognitive psych papers not report demographics. One of my old professors justified it by saying they weren't necessary because the work was just focused on brains and not culture.

I mean, I totally get it matters more in some cases than others. I'm way less worried when it comes to a study of dynamic scene perception in infants than I am in a study of affective responses to the COVID-19 pandemic. I've seen reviewers slam studies for slight under-representation of group X according to national statistics (e.g., low-but-not-zero Hispanic sample in a region with low Hispanic population) in studies where there was zero reason to believe that would impact things.

However you always-always-always report your sample demographics in as much detail as you can. And as someone who has done studies examining cultural influences on the brain, that blanket justification reveals a fundamental lack of understanding of both culture and brains.

 

[futureapppsy2]

I mean, I totally get it matters more in some cases than others. I'm way less worried when it comes to a study of dynamic scene perception in infants than I am in a study of affective responses to the COVID-19 pandemic. I've seen reviewers slam studies for slight under-representation of group X according to national statistics (e.g., low-but-not-zero Hispanic sample in a region with low Hispanic population) in studies where there was zero reason to believe that would impact things.

However you always-always-always report your sample demographics in as much detail as you can. And as someone who has done studies examining cultural influences on the brain, that blanket justification reveals a fundamental lack of understanding of both culture and brains.

You see the same logic in ABA articles, at least historically--the idea that culture doesn't matter because the principles of behavior are universal. There's been several articles, including one by me and some colleagues, calling that out, and it's slowly changing.

 

[ClinicalABA]

I mean, I totally get it matters more in some cases than others. I'm way less worried when it comes to a study of dynamic scene perception in infants than I am in a study of affective responses to the COVID-19 pandemic. I've seen reviewers slam studies for slight under-representation of group X according to national statistics (e.g., low-but-not-zero Hispanic sample in a region with low Hispanic population) in studies where there was zero reason to believe that would impact things.

However you always-always-always report your sample demographics in as much detail as you can. And as someone who has done studies examining cultural influences on the brain, that blanket justification reveals a fundamental lack of understanding of both culture and brains.

This is something Roberts points out in his more recent paper. There is an assumption that white=default. If the sample is white, the study is presented/titled as something along the lines of "Effects of Priming on Memory". If the sample is Asian, the study is presented/titled "Memory in Asians: the Effects of Priming." This establishes "white" as the norm, and Asian as a "special pipulation" that needs to be specifically identified. Even the the majority of people on this earth are technically "Asian", the WEIRD sample is treated as the default human condition and not specifically identified as "Memory in Caucasians: the Effects of Priming." In their original 2020 article, Robert's et al. cite research showing that culture can influence cognitive processes, thus assuming white=default or norm is erroneous, scientifically speaking.

 

[ClinicalABA]

You see the same logic in ABA articles, at least historically--the idea that culture doesn't matter because the principles of behavior are universal. There's been several articles, including one by me and some colleagues, calling that out, and it's slowly changing.

And this is important research that needs to keep happening. Idiogrqphic assessment that accounts for cultural, gender, SES, sexuality, and family specific contingencies is key not only in developing effective interventions, but also (and probably more importantly) appropriate treatment goals.

In my grad program we rotated our treatment teams annually around different theories and perspectives. I was struck by my experiences on the Social Consructivist team more closely mirrored my Behavioral Therapy perspective than some of the other team (e.g. psychodynamic) that assumed cross cultural consistencies. One of the things that draws me to GOOD ABA is that no variable/contingency/experience is given a default status of being more universally important, default, than any other. That's how I try to teach my students (when I actually show up for class!). It can be very difficult, being raised, accultured, and phentypically/socially identified as a cisgendered white male, to counteract my biases, but I try.

 

[borne_before]

What if racial differences were found in cog lit? Could you just imagine the **** storm on the woke side and hard on for the cherry picking racist side.

 

[WisNeuro]

What if racial differences were found in cog lit? Could you just imagine the **** storm on the woke side and hard on for the cherry picking racist side.

Oh, we've already hit that ****storm throughout the decades, most recently with the NFL settlements and Heaton norms.

 

[ClinicalABA]

What if racial differences were found in cog lit? Could you just imagine the **** storm on the woke side and hard on for the cherry picking racist side.

They have been (c.f. Roberts et al. 2020).

 

[PsychPhDone]

I keep trying to get papers focused on women into "mainstream" journals and they get desk rejected, editors saying they should go in women's journals, as if you must mentally prepare yourself for seeing an article about menopause. Not like it affects half the population.

Anyway just a side-tracking comment about the view of "default" in the academy.

 

[WisNeuro]

I keep trying to get papers focused on women into "mainstream" journals and they get desk rejected, editors saying they should go in women's journals, as if you must mentally prepare yourself for seeing an article about menopause. Not like it affects half the population.

Anyway just a side-tracking comment about the view of "default" in the academy.

If you're also looking at the cognitive aspects of it, some of the neuro journals may be more receptive. There's been a lot of interest in that in recent years.

 

[PsychPhDone]

If you're also looking at the cognitive aspects of it, some of the neuro journals may be more receptive. There's been a lot of interest in that in recent years.

I'm not but this could be a fun lil direction to go in at some point. Would need a neuro collaborator though. 😉 Will keep in mind!