Contest journal decision?

[MCParent]

I wanted to see what others thought before I decided what I'm going to do on this. Here's the sitch:

-Asked to review paper. Paper is on niche aspect of a major research area I work on. Not bragging, but I might be one of ten people in the country who fully know this topic area. Just to say, review is on a topic I know backward and forward.
-Paper is very weak. Qual, no coding, no interrater reliability. No participant table, and quotes are not attributed. Huge editorial leaps. Method is essentially one step above journalism. Author justifies this saying it's how the method works. Not true; other papers using this method have participant tables. I'll give them that interrater agreement is often not done with this method. Which I said doesn't mean you don't have to do coding, it means your method is weak. (There was nothing amazingly special about the sample or data that means coding could not have been done). Huge problems in the paper, from a weak 2-page intro to mistakes in interpreting data (e.g., saying participants were "low on (x)".... there's no control group and they asked about (x) directly, os obviously everyone mentioned it. And then, no coding = no counts for occurrence of mention of (x).... All of this is in my first review, and not addressed in the revision.
Round 1-R1 says reject, I say reject, R3 says accept w a perfunctory review.
Round 2-R1 not invited or declines re-review, I say reject as basically all problems persisted, R3 says same thing they said first round. Paper is accepted.

What would y'all do? Considering reporting this to APA, since it seems to me the paper failed peer review and has major flaws but is being published anyway. Ordinarily I might not care so much, but this is a niche I have put energy into and frankly the paper detracts from the body of work. Journal is open access, so TBH I am also worried that affected the decision.

---

Members don't see this ad.

 

[NotTheHoff]

Three options jumped out to me, probably in the order I'd consider them:

1. A side "WTF" email to the editor: the formality of it would depend on my familiarity with them.

2. Drafting a response paper that highlights your concerns. You could check with the editor if they'll consider it in some form.

3. Is there a journal policy on reviewers posting/de-anonymizing their reviews? You could host it on PsyArXiv.

 

[MCParent]

I don't know the editor at all. That would also be my first call, usually. But I suppose since I'm making essentially an ethics complaint I should email the editor first. Good point.
It's not really the normal situation where's I'd want to write a response (which I've done before, when a journal asked me to). The issues are pretty solidly methodological, as opposed to conceptual.
I wasn't familiar with PsyArXiv, thanks; I'll check it out.

 

[Ollie123]

Is this an APA journal? Didn't think they had any straight open access journals but I've moved towards neuroscience journals and more biomedical journals in recent years so don't know. Otherwise, I have little idea what they could do. If no, how legit is the publisher? Big difference between say - BMC/PLoS/JAMA Open and the random "open access" versions attached to some already barely-respected journals I'm seeing now that I think exist basically as a way for them to still profit off rejections. That's not even to speak of all the true trash/predatory stuff out there, but assume you aren't reviewing for those.

I've been in the same boat where I've repeatedly recommended rejection for something that kept coming back. It was clear I wasn't the oddball either because the other reviews were consistent with mine. Sometimes there have been legit reasons (i.e. it was controversial and the editor wanted it out there as a provocative thought-piece/counterpoint or the topic was new enough they were willing to overlook some gaping methodological concerns - I disagreed, but fair enough). Other times not. All I've found I can do is refuse to review for that journal/editor anymore and not be shy about spelling out the reason why if they ask.

It is much too easy to publish these days. I think its part of the reason tenure expectations have gotten out of control.

 

[MCParent]

Is this an APA journal? Didn't think they had any straight open access journals but I've moved towards neuroscience journals and more biomedical journals in recent years so don't know. Otherwise, I have little idea what they could do. If no, how legit is the publisher? Big difference between say - BMC/PLoS/JAMA Open and the random "open access" versions attached to some already barely-respected journals I'm seeing now that I think exist basically as a way for them to still profit off rejections. That's not even to speak of all the true trash/predatory stuff out there, but assume you aren't reviewing for those.

I've been in the same boat where I've repeatedly recommended rejection for something that kept coming back. It was clear I wasn't the oddball either because the other reviews were consistent with mine. Sometimes there have been legit reasons (i.e. it was controversial and the editor wanted it out there as a provocative thought-piece/counterpoint or the topic was new enough they were willing to overlook some gaping methodological concerns - I disagreed, but fair enough). Other times not. All I've found I can do is refuse to review for that journal/editor anymore and not be shy about spelling out the reason why if they ask.

It is much too easy to publish these days. I think its part of the reason tenure expectations have gotten out of control.

Taylor and Francis. IF is weak, about 1. Editor is decently known, AEs are not. I'd never actually heard of the journal, am not on the board, etc. Accepted invite bc of paper topic. Which irks me even more, why waste my time if you aren't going to use my review?

 

[Ollie123]

Hmm. Legit publisher but could be one of these profiteering "spinoff" journals. Can you PM me the name? I'm curious now.

Based on all this, its probably just a ****ty journal. I'm not sure what recourse you could have over editorial decisions at some bottom-tier journal. IF=1 usually means they publish anything. I've been caught in the same mess and was also annoyed when I felt the detailed review I wrote after slogging through some barely-coherent junk science got ignored. Email the editor if its cathartic, but I'd just refuse to review for them again, never send them anything, discourage colleagues from sending them anything and move on. Only way I might feel differently is if it could be used to actively promote harmful clinical practice (i.e. advocating for ineffective treatments versus just some esoteric theoretical paper that could conceivably cause problems down the line).

 

[WisNeuro]

Hmm. Legit publisher but could be one of these profiteering "spinoff" journals. Can you PM me the name? I'm curious now.

Based on all this, its probably just a ****ty journal. I'm not sure what recourse you could have over editorial decisions at some bottom-tier journal. IF=1 usually means they publish anything. I've been caught in the same mess and was also annoyed when I felt the detailed review I wrote after slogging through some barely-coherent junk science got ignored. Email the editor if its cathartic, but I'd just refuse to review for them again, never send them anything, discourage colleagues from sending them anything and move on. Only way I might feel differently is if it could be used to actively promote harmful clinical practice (i.e. advocating for ineffective treatments versus just some esoteric theoretical paper that could conceivably cause problems down the line).

Do you see any downside to indicating to the Editor that if this paper gets published, that you'd like to be removed from any future consideration in reviewing articles for this journal secondary to this issue? In MCP's case, I'd imagine he's built up enough of a good reputation that he'd suffer no real repercussion or political fallout. Theoretically, any issue for a an ECP in this situation?

 

[MCParent]

Do you see any downside to indicating to the Editor that if this paper gets published, that you'd like to be removed from any future consideration in reviewing articles for this journal secondary to this issue? In MCP's case, I'd imagine he's built up enough of a good reputation that he'd suffer no real repercussion or political fallout. Theoretically, any issue for a an ECP in this situation?

This is a good idea, yup. I can see where someone else might not be able to swing this tho.

Hmm. Legit publisher but could be one of these profiteering "spinoff" journals. Can you PM me the name? I'm curious now.

Based on all this, its probably just a ****ty journal. I'm not sure what recourse you could have over editorial decisions at some bottom-tier journal. IF=1 usually means they publish anything. I've been caught in the same mess and was also annoyed when I felt the detailed review I wrote after slogging through some barely-coherent junk science got ignored. Email the editor if its cathartic, but I'd just refuse to review for them again, never send them anything, discourage colleagues from sending them anything and move on. Only way I might feel differently is if it could be used to actively promote harmful clinical practice (i.e. advocating for ineffective treatments versus just some esoteric theoretical paper that could conceivably cause problems down the line).

This is useful thanks. If it were something directly related to clinical practice I'd be even more mad. As it stands I'm mostly mad bc the paper is in a small area, and so one **** paper has more influence.

 

[Ollie123]

I mean, academia is a small world so its pretty easy to come up with scenarios where anything comes back to bite someone. Said editor could be super-egotistical and take this very personally and then be on an NIH study section reviewing MCPs next grant application. Or action editor on a paper for another journal that Mike has under review right now and decide to nitpick every little methodological flaw. That's probably unlikely unless the email is phrased in a pretty aggressive/explicit manner ("I didn't realize this was a predatory journal that published anything regardless of scientific merit - please do not ask me to review for you again"). If its a little more polite/professional, odds of consequences go down signfiicantly

Those may or may not be meaningful consequences to someone at MCP's level though. Could it "conceivably" cause an annoyance down the line? Sure. Certainly not going to ruin his career. He might have tenure by now anyways?

 

[MCParent]

He might have tenure by now anyways?

Yup! Word came down last month.