A nursing home worker receives the first Pfizer-BioNTech vaccine administered in the Netherlands in January. A Dutch database is at the heart of a controversy surrounding a new paper on COVID-19 vaccine safety.
Piroschka van de Wouw/Pool via Reuters
Science’s COVID-19 reporting is supported by the Heising-Simons Foundation.
Several reputed virologists and vaccinologists have resigned as editors of the journal Vaccines to protest its 24 June publication of a peer-reviewed article that misuses data to conclude that “for three deaths prevented by [COVID-19] vaccination, we have to accept two inflicted by vaccination.”
Since Friday, at least six scientists have resigned positions as associate or section editors with Vaccines, including Florian Krammer, a virologist at the Icahn School of Medicine at Mount Sinai, and Katie Ewer, an immunologist at the Jenner Institute at the University of Oxford who was on the team that developed the Oxford-AstraZeneca COVID-19 vaccine. Their resignations were first reported by Retraction Watch.
Related
“The data has been misused because it makes the (incorrect) assumption that all deaths occurring post vaccination are caused by vaccination,” Ewer wrote in an email. “[And] it is now being used by anti-vaxxers and COVID-19-deniers as evidence that COVID-19 vaccines are not safe. [This] is grossly irresponsible, particularly for a journal specialising in vaccines.”
The paper is a case of “garbage in, garbage out,” says Helen Petousis-Harris, a vaccinologist who directs the Vaccine Datalink and Research Group at the University of Auckland and who also resigned as a Vaccines editor after reading the paper. Diane Harper, an epidemiologist at the University of Michigan, Ann Arbor, who was founding editor-in-chief of Vaccines, also resigned, as did Paul Licciardi, an immunologist at Murdoch Children’s Research Institute in Parkville, Australia, and Andrew Pekosz, a respiratory virologist at the Johns Hopkins University Bloomberg School of Public Health.
The resignations began Friday, the day after the paper was published. By early Monday, Fanny Fang, the journal’s managing editor, wrote to the editorial board members that Vaccines—a reputable open-access journal launched in 2013 by Basel, Switzerland–based publisher MDPI—had opened an investigation into the paper. “We are treating this case with the utmost seriousness and are committed to swiftly correcting the scientific record,” she wrote.
Later on Monday, Vaccines’s editors published an Expression of Concern about the paper. “The major concern is the misrepresentation of the COVID-19 vaccination efforts and misrepresentation of the data,” they wrote.
The paper has drawn nearly 350,000 readers as of 1 July and has been tweeted by antivaccination activists with hundreds of thousands of followers.
None of the paper’s authors is trained in vaccinology, virology, or epidemiology. They are: Harald Walach, a clinical psychologist and science historian by training who describes himself as a health researcher at Poznan University of Medical Sciences in Poland; Rainer Klement, a physicist who studies ketogenic diets in cancer treatment at the Leopoldina Hospital in Schweinfurt, Germany; and Wouter Aukema, an independent data scientist in Hoenderloo, Netherlands.
The three peer reviewers on the paper, two of them anonymous, did not offer any substantial criticism of the authors’ methodology in these brief reviews. One of them, Anne Ulrich, a chemist who directs the Institute of Biological Interfaces and is chair of biochemistry at the Karlsruhe Institute of Technology in Germany, wrote that the authors’ analysis “is performed responsibly … and without methodological flaws … and the results were interpreted with the necessary caveats.”
Ulrich reiterated that view in a 1 July email to ScienceInsider: “The analysis by Walach et al. was done in my opinion responsibly and without flaws,” she wrote.
One of the anonymous reviewers wrote that the manuscript “is very important and should be published urgently,” offering almost no other comment.
Vaccines did not respond to a list of questions, including how the peer reviewers were chosen.
Petousis-Harris says: “It’s very evident from their reviews that they don’t have any topic expertise. The authors don’t either. It’s a bit remiss when you don’t bring any of a large body [of expert vaccine safety scientists] into your study and manuscript review.”
To draw their conclusions, the paper’s authors computed COVID-19 deaths prevented by vaccines by using data from a study of 1.2 million Israelis, half of whom received the Pfizer-BioNTech vaccine and half of whom did not. They estimated that 16,000 people needed to be vaccinated to prevent one COVID-19 death—a measure that is problematic, critics say, because as a vaccine succeeds, the number of people that must be vaccinated to prevent a death grows ever-larger.
To compute deaths “caused” by vaccine side effects, they used EU data on the number of vaccine doses delivered in the Netherlands and data from that country’s national database for adverse drug reaction reporting, which was more voluminous than registry data from other EU countries.
The Netherlands Pharmacovigilance Center, called Lareb, is similar to the United States’s VAERS reporting system, in which anyone can file a report of an adverse event that occurs after vaccination; the existence of a report in the database does not prove a vaccine caused an adverse event or death. Such databases are not used to systematically assess vaccine risks, but to search for early signals of rare but real vaccine side effects, such as the clotting disorder associated with some COVID-19 vaccines, that can then be followed up with systematic study.
The website of the Dutch registry clearly notes its reports do not imply causality. But the authors reported that, using it, they found “16 serious side effects per 100,000 vaccinations and the number of fatal side effects is at 4.11/100,000 vaccinations. For three deaths prevented by vaccination we have to accept two inflicted by vaccination.”
On 25 June, the day after the paper’s publication, Lareb’s head of science and research, Eugène van Puijenbroek, sent an email to Vaccines’s editors, criticizing the paper and requesting a correction or retraction.
“A reported event that occurred after vaccination is … not necessarily being caused by the vaccination, although our data was presented as being causally related by the authors,” van Puijenbroek wrote. “Suggesting all reports with a fatal outcome to be causally related is far from truth.”
He also took the authors to task for stating in the paper that “the Dutch [registry] data, especially the fatal cases, were certified by medical specialists.”
“This point is simply incorrect,” van Puijenbroek wrote. “The authors seem to refer to [Lareb’s] policy plan. However, in this plan (in Dutch), it is nowhere mentioned that the reports are ‘certified’ by medical specialists.”
In an email to ScienceInsider, van Puijenbroek also worried that the paper will erode public willingness to report to the Dutch registry. “[People who report adverse events to us] have to be sure that the data they are contributing will be used in a correct and scientific sound way. Articles like this undermine this confidence.”
Walach, the first author, says he stands by the paper. He adds that the clinical trials of the vaccines that are now authorized for emergency use, which involved tens of thousands of people, “were not big enough or long enough to really see whatever safety concerns you will have.”
Aukema adds that he’s pleased about the controversy, because it means “the [COVID-19 vaccine safety] topic is getting a lot of attention. I’m happy about that.”
The three authors submitted a rebuttal to Vaccines’s expression of concern on 29 June. In it, they write that they interpreted Lareb’s public statements “to mean that those reports that are obviously without any foundation are taken out such that the final data-base is at least reliable to some degree.”
The authors also insist that they did not imply that the side effects reported in the Dutch registry were necessarily caused by COVID-19 vaccines. “Currently we only have association, we agree, and we never said anything else.”
Petousis-Harris says she has been pleased with what she sees as a speedy and serious response from the leadership at Vaccines, and that her resignation may not be permanent.
“It has been a good, solid, respectable journal. And if [the paper] is going to be retracted … then I would still be happy to part of the team.”
Source: Science Mag