Back in January, an academic study gave heart to critics of COVID-19 vaccines by estimating the number of U.S. deaths from the vaccines at 278,000.
That was a bombshell, if true. The Centers for Disease Control and Prevention cited only 19,476 reports of deaths after COVID vaccination in a national database of unverified adverse reactions to the shots.
Since even that surely inflated figure, which reflects an unknown number of deaths from unrelated causes, amounted to less than three-thousandths of a percent of the 672 million doses of COVID vaccines administered in the U.S., the CDC properly judged the shots "safe and effective" and severe post-vaccine reactions "rare."
Yet the study, by economist Mark Skidmore of Michigan State University, was taken as gospel truth by a legion of anti-vaccine activists.
And why not? It had been published in BMC Infectious Diseases, a peer-reviewed medical journal associated with the Nature publishing group, which lent it a gilt-edged luster. Indeed, it ranked as the most-viewed paper in the journal's history.
Now take a deep breath. BMC Infectious Diseases retracted the Skidmore study on Tuesday, specifically citing doubts about "the validity of the conclusions" related to death statistics because of flaws in its methodology. Skidmore disagrees with the retraction.
The retraction, which followed months of dickering between Skidmore and the journal's editor over the nature and text of the retraction notice, points to some important questions about how the spread of misinformation about COVID affects public health.
It also raises questions about how the piece got published in the first place. As Stephanie M. Lee of the Chronicle of Higher Education has pointed out, the flaws in Skidmore's paper were virtually self-evident from the moment it reached print.
Veteran pseudoscience debunker David Gorski identified them within a day of its official publication, calling the paper "antivax propaganda disguised as a survey," noting Skidmore's record of anti-vaccine commentary, and asking: "How on earth did BMC Infectious Diseases publish such dreck?"
The journal hasn't answered that question. It did, however, publish the comments of the two peer reviewers of Skidmore's paper, one of whom wished to remain anonymous.
That reviewer raised a few doubts about Skidmore's data but called the paper "a fantastic study anyway." The second reviewer, Sudanese public health researcher Yasir Ahmed Mohammed Elhadi, initially recommended against publication but changed his mind after the paper was revised.
Doubts about scientific papers related to COVID-19 have been rife during the pandemic. The database of retractions of COVID studies is up to 313 papers at the invaluable blog Retraction Watch.
That matches the average retraction rate in all fields, says Ivan Oransky, one of the blog's creators. But since COVID has been a topic of study for only three years, it suggests that those papers are getting especially intense scrutiny.
"It makes sense to prioritize cleaning the scientific record of papers that could have an impact on public health," Oransky says.
As for how papers like Skidmore's make it through peer review, that only underscores that our faith in peer review as a bulwark against sloppy or even dishonest research is misplaced.
"Retractions of peer-reviewed papers happen thousands of times a year," Oransky says, "and those are the ones we know about." In some cases it can take years for a journal to retract a flawed paper.
In the Skidmore case, "some public scrutiny and attention has prompted speedier action" by the publishing journal, Oransky says. "But we shouldn't be surprised anymore that flawed papers make it through peer review because it happens every day."
There are few fields in which flawed scientific research has been weaponized as much as in studies of COVID treatments. Worthless nostrums such as hydroxychloroquine and ivermectin have been promoted based on transparently bogus evidence.
COVID vaccines have been an especially common target. Among the promoters of questionable anti-vaccine research are Florida Surgeon General Joseph Ladapo and his political patron, Florida Gov. Ron DeSantis.
In October, Ladapo recommended against administering the Moderna or Pfizer mRNA vaccines to males ages 18 to 39. He based his guidance on a study purporting to show an elevated risk of cardiac-related death among males in that age range within 28 days of receiving the vaccines.
Numerous qualified professionals pointed out, however, that the study was so sloppy and incoherent that it amounted to pseudoscience.
In the last few days, as it happens, the Tampa Bay Times revealed that data showed that catching COVID itself was much more dangerous to cardiac health than taking the vaccines. That information was stripped out of the state study before its final publication, according to drafts of the study the newspaper obtained via a public records request.
That brings us back to the Skidmore study.
Superficially, the study was aimed at determining the factors that led people to be more or less amenable to taking a COVID vaccine. Based on an anonymous database of 2,840 respondents compiled by a third-party survey firm, Skidmore found that people who knew someone who had a serious bout of COVID illness were more likely to get vaccinated, and knowing someone who appeared to have suffered a post-vaccination injury was associated with being less likely to take the shot.
As far as that goes, it's not especially surprising. But Skidmore went further. He extrapolated from the number of respondents who said they knew someone who had died from the vaccine to conclude that the number of U.S. vaccine-related deaths "may be as high as 278,000."
It should be obvious that this extrapolation was groundless. His base figures on vaccine-related deaths came from people who had no way of knowing that the illness or death their acquaintances suffered after vaccination had anything to do with the shot.
In an email, Skidmore defended his method. "Reporting on respondent feedback from surveys on health issues is not uncommon, and generating projections from a valid sample is a standard statistical technique that is commonly used in survey research," he told me. "The research approach is valid. I stand by the methods, analysis, and conclusions."
Skidmore is right, up to a point. In a previous study, he analyzed the social contexts of prescription drug abuse using similar methods. The problem here is that he used people's perceptions about whether acquaintances died as a result of their COVID vaccinations to estimate the actual number of deaths.
That's a big leap, and nothing like a valid method. In a lengthy exchange of correspondence with Adrian Gonzalez Lopez, the editor of BMC Infectious Diseases, Skidmore acknowledged that he had no way of determining whether the study subjects' perceptions about vaccine-related injuries were accurate.
"Validating reported fatalities is not possible with an anonymous survey," he stated in the course of the exchange, which he has made public.
Skidmore noted that his article "clearly acknowledges ... that the reports are based on respondent perceptions and calls for further research, which can either verify or refute the findings.
There were other red flags in Skidmore's paper. One was his reliance on the CDC's Vaccine Adverse Events Reporting System, or VAERS, as a benchmark database for COVID vaccine injuries.
VAERS is not suitable for that purpose, as CDC makes crystal clear. That's because it's a repository of entirely voluntary reports that can be filed by anyone, including but not limited to doctors, patients and family members, who aren't required to document the injury.
"VAERS is not designed to determine if a vaccine caused or contributed to an adverse event," the CDC says. "A report to VAERS does not mean the vaccine caused the event."
What anecdotal claims about post-vaccine illnesses and death often overlook is the law of large numbers: With 672 million doses administered, "by random chance alone and the law of large numbers, there will be a large number of people who die within, say, 30 days of being vaccinated even if the vaccine has absolutely nothing to do with their deaths," Gorski observes. "That's why VAERS numbers, divorced from baseline rates, are utterly meaningless."
The journal editor, Gonzalez Lopez, had also raised a question with Skidmore about Catherine Austin Fitts, whom Skidmore had identified as the source of funding for his survey. Gonzalez Lopez noted that Fitts "has made regular statements regarding COVID-19 vaccination and is a former politician," and therefore her role should have been "disclosed as a conflict of interest."
In fact, Fitts is known as a critic of COVID vaccination. She's a financier and former housing official in the George H.W. Bush administration who has promoted the anti-vaccine campaign of Robert F. Kennedy Jr.
In a review of Kennedy's 2021 book attacking Anthony S. Fauci, who recently retired as director of the National Institute of Allergy and Infectious Diseases, she wrote of Fauci — a common target of anti-vaccine fanatics — that "Anthony Fauci and his colleagues are guilty of crimes against humanity."
According to a 2021 article in the Washington Post, Fitts appeared in an anti-vaccine video asserting falsely that "the coronavirus vaccine is 'full of these mystery ingredients' and will 'modify your DNA and for all we know make you infertile.'"
Skidmore refused to augment his disclosure about Fitts' role. "I have appropriately acknowledged Catherine Austin Fitts as the source of funding for the survey," he told the editor. "Accordingly, I do not consider non-competing financial interests to exist."
The journal ultimately dropped a reference to Fitts from its proposed retraction notice. But that doesn't change how we should think about papers like Skidmore's.
Its effect was to muddy the waters of vaccine safety by positing hard figures for COVID vaccine deaths based on no empirical evidence. It came from an author whose personal blog bristles with anti-vaccine commentary.
BMC Infectious Diseases was right to retract the paper, but it should have done so even more expeditiously. Why did it take so long?
____
