E. Petersen/Science
Science’s COVID-19 reporting is supported by the Pulitzer Center and the Heising-Simons Foundation.
Three months after retracting a high-profile COVID-19 paper, editors at The Lancet hope to assure the research community that they’ve learned their lesson. The journal yanked a study on risks of hydroxychloroquine—an antimalarial drug whose proposed use as a COVID-19 treatment has stirred scientific and political controversy—in June when its authors couldn’t prove the underlying patient data even existed. Yesterday, it announced policies, effective immediately, that aim to keep flawed studies using “large, real-word datasets” from slipping past peer review again. They include stricter standards for the expertise of peer reviewers of such papers and requirements that all authors vouch for the validity of their data and detail their data-sharing plans.
“We aim to learn whenever we can how we might reduce risks and improve processes,” the editors wrote in an accompanying editorial about stresses the pandemic has placed on peer review.
Related
The now-retracted study, published in May, relied on data from the small Chicago-based company Surgisphere, which purported to have collected and analyzed patient records from hundreds of hospitals around the world. But the paper’s central claim—that antimalarial drugs increased COVID-19 patients’ risk of death—quickly unraveled as observers questioned the study’s large sample size and details about patient demographics and dosing. The Lancet retracted the paper when Surgisphere declined to make underlying data available for an independent audit. The same day, The New England Journal of Medicine (NEJM) retracted another study using Surgisphere data, which found that certain blood pressure drugs didn’t increase risk of death from COVID-19.
The Lancet’s new standards appear designed to plug specific holes in the review process that let the Surgisphere results through. That paper was published despite the fact that only one of the four authors, Surgisphere founder Sapan Desai, had accessed the primary data. (His co-authors, cardiologist Mandeep Mehra of Harvard University and Brigham and Women’s Hospital, Frank Ruschitzka of University Hospital Zürich, and Amit Patel, an adjunct faculty member at the University of Utah, announced in June that they could “no longer vouch for the veracity” of the data after a plan for an independent audit apparently fell apart.) The Lancet will now require authors to certify that more than one of them has accessed and verified the study’s underlying data—and indicate which authors have done so. In collaborations between academic labs and commercial companies, one of the academic authors must certify that they’ve seen the data.
The journal will also require authors of all of its papers to detail what data they will share, the criteria for access, and how they’ll make them available. That mandate has previously applied only to reports of clinical trials, not retrospective studies like the Surgisphere paper. (NEJM hasn’t published a similar statement about changes in peer review practices but its editor told The New York Times that the journal should have used reviewers familiar with large hospital data sets.)
For any study involving a large patient data set, The Lancet says it will now ensure that at least one reviewer “can understand and comment on its strengths and limitations.” It will bring in experts in both statistics and data science for studies involving “very large datasets.” And it will ask reviewers specifically whether they have “concerns about research integrity or publication ethics” of the paper.
The changes are unlikely to silence criticism of the journal in the wake of the retraction.
“This is not enough,” Sweden-based human-computer interaction and scientific visualization researcher Lonni Besançon Tweeted in response to the announcement. The journal needs to ensure that data underlying its studies can be checked—at least by an independent third party, he wrote.
Large prospective studies have now shown no benefit to hydroxychloroquine, but they have not replicated the finding of increased risk of death reported in the retracted Lancet paper. The retraction and the journal’s efforts at reform haven’t contained the spread of misinformation from publishing the Surgisphere results. A 16 September report from the French National Authority for Health cites the Lancet paper among the reasons it revoked its recommendation for the emergency use of hydroxychloroquine, without noting that the paper has since been discredited.
Source: Science Mag