ROBERT NEUBECKER
By Christie Wilcox
There’s a running joke in academia about Reviewer 2. That’s the reviewer that doesn’t bother to read the manuscript a journal has sent out for evaluation for possible publication, offers condescending or outright offensive comments, and—of course—urges the irrelevant citation of their own work. Such unprofessional conduct is so pervasive there’s even a whole Facebook group, more than 25,000 members strong, named “Reviewer 2 Must Be Stopped!” But it is no laughing matter, concludes a new study that finds boorish reviewer comments can have serious negative impacts, especially on authors belonging to marginalized groups.
Peer reviewers are supposed to ensure that journals publish high-quality science by evaluating manuscripts and offering suggestions for improvement. But often, referee comments stray far from that mission, found the new PeerJ study, which surveyed 1106 scientists from 46 countries and 14 disciplines. More than half of the respondents—who were promised anonymity—reported receiving at least one “unprofessional” review, and a majority of those said they had received multiple problematic comments.
Those comments tended to personally target a scientist, lack constructive criticism, or were just unnecessarily harsh or cruel, the authors report. For example, one author received a review that stated: “The phrases I have so far avoided using in this review are ‘lipstick on a pig’ and ‘bullshit baffles brains.’” Another reported receiving this missive: “The author’s last name sounds Spanish. I didn’t read the manuscript because I’m sure it’s full of bad English.”
“It wasn’t like it was just a certain group receiving these comments—everybody was getting them,” says ecologist Amber Stubler of Occidental College in Los Angeles, California, a co–lead author of the study. “That is really very disturbing in and of itself.”
What wasn’t equal was the toll these reviews took on the respondents. White men reported being “the least impacted by the unprofessional peer reviews,” says co–lead author Nyssa Silbiger, an ecologist at California State University in Northridge. But women, nonbinary individuals, and people of color all were more likely to report that unprofessional reviews increased feelings of self-doubt and harmed their scientific productivity. People of color were also more likely to say the reviews delayed their career advancement.
Those reports are not surprising, psychologist Denise Sekaquaptewa of the University of Michigan in Ann Arbor wrote to ScienceInsider in an email. They line up with a lot of findings in the psychological literature on stereotype threat—the psychological harm caused by pervasive negative stereotypes. Essentially, because there are stereotypes that women or people of color are less intelligent or scientifically minded, receiving a review that reinforces such stereotypes—no matter how inaccurate—can create psychological distress. That distress, in turn, can result in self-doubt, impaired performance, and delayed career advancement.
Those are issues that microbial ecologist Adriana Romero-Olivares, a postdoc at the University of New Hampshire in Durham, can relate to. In an essay published in Science, she wrote about receiving a particularly negative review, on the first paper she submitted to a journal, attacking her ability to write in English. (Romero-Olivares is from Mexico.) “I just didn’t understand why they were criticizing my English and not focusing on the science,” she recalls. The comments “did really make me question [myself],” she recalls, although she says she has since grown a “tough skin.”
The authors of the new study say they hope it will help spur discussion of ways to curb unprofessional comments. And researchers already have some ideas. For example, Linda Beaumont, a climate researcher at Macquarie University in Sydney, Australia, views the problematic reviews she has received as “another form of bullying” that should be called out. That’s why she’s in favor of publishing peer-review comments, which are typically kept private, alongside the final paper, a practice already standard for some journals. Silbiger and Stubler note that their survey respondents said few—less than 3%—of the unprofessional reviews involved papers submitted to journals with such policies.
Other researchers simply want to lift the cloak of anonymity generally given to referees, either throughout the review process or at the end of it. Such “open peer review” has been shown to increase the quality and professionalism of reviewer comments.
But Romero-Olivares and other researchers believe revealing reviewer identities will disproportionately harm early career scientists because it exposes them to retaliation from more senior colleagues displeased with their critiques. She’d prefer that all journals use in double-blind peer review, in which the reviewers don’t know the identities of the authors, and vice versa. Some scientists believe such a system would reduce outright bigotry.
Silbiger and Stubler believe journal editors should feel empowered to reject reviews that are inappropriate and even refuse to work with problematic reviewers. But they note it can be hard for an editor to argue that a particular reviewer has crossed a line because few journals have explicit guidelines for referees, except for stating that their comments should be “professional.” To solve that problem, Beaumont has called on journals to adopt specific codes of conduct for reviewers. Ideally, Silbiger says, those codes also would specify “serious consequences” for violations.
In the meantime, Sekaquaptewa would like to see researchers go a step further in studying the impacts of poor reviewer behavior. In particular, she says a deeper dive into the effects of both generally nasty comments, and ones that specifically target race, gender, or ethnicity, could be useful. That’s something Silbiger and Stubler are hoping to look into. The current study, Stubler says, “sort of scratches the surface.”
Source: Science Mag