The Occupational Safety and Health Administration’s controversial plan to enforce COVID-19 vaccinations for large businesses — recently enjoined by the Fifth Circuit Court of Appeals — was ostensibly designed to minimize “deadly outbreaks of COVID-19.” The ability of COVID-19 vaccines to prevent transmission and protect life is at the heart of the OSHA mandate and the fierce debate over similar mandates now embroiling much of the world.
Nearly 18,000 scientific papers have been published since last year on COVID-19 and vaccines, so the task of sifting through the evidence to help critically evaluate whether vaccines reduce risk of transmission and death seems daunting. It turns out, though, that two studies stand far above the rest in terms of rigor and quality.
These two studies, which are a continuation of the trials that the FDA required for emergency use approval, were published last month in the New England Journal of Medicine. They are fundamentally distinct from the other studies in that they are the only clinical trials yet reported to randomize adults to receive either a COVID-19 vaccine (Pfizer or Moderna) or a placebo injection, and then follow them over time. Why is this important? Because the blinded randomized controlled study design they used is the gold standard and most rigorous scientific tool available to examine cause and effect relationships between an intervention and outcome.
This design also limits as much as possible the influence of other factors, whether known or unknown, that could affect the outcome. Many studies have used other designs to try and understand how well the vaccine protects against COVID-19, but no matter how well planned or executed, none of these studies approaches the level of scientific rigor that a well-conducted double blind randomized controlled trial offers.
So did these two clinical trials find that vaccination reduced the risk of dying from COVID-19? No, they did not. They were not designed to do that. Both studies report high efficacy at reducing symptomatic Covid disease, somewhere in the 89%-95% efficacy range, as well as serious disease, in the range of 80%-100%. They did not enroll enough older high-risk people to have sufficient sample size to determine if the vaccines also reduce death from COVID-19.
The Moderna study reported one death from COVID-19 in the vaccinated group and three in the unvaccinated group, far too few to make any statistical conclusion. The Pfizer trial was even more inconclusive because the findings published in the New England Journal report (one COVID-19 death in the vaccinated group and two in the unvaccinated group) differed from what Pfizer later reported to the Food and Drug Administration, and the FDA update did not specify the number of COVID-19 deaths.
In addition to Covid deaths, the studies also evaluated all-cause mortality, which counts every death that occurred during the study period. That gives a larger sample size. All-cause mortality is an outcome of interest not simply because it circumvents the oftentimes subjective decision as to why someone died but also because it balances all the possible effects of a COVID-19 vaccine, both good and bad, that could influence risk of death. In other words, it allows us to quantify lives saved by the COVID-19 vaccine while taking into account potential lives lost from vaccine-related heart disease, blood clots, severe allergic reactions, and perhaps other causes.
Because results from the two trials were so similar, regardless of the type of vaccine used, it is helpful to merge the results. Following a combined total of 74,580 individuals — half given the COVID-19 vaccination and half given a placebo shot, over six to seven months — the two studies reported that thirty-seven people who were vaccinated died as compared to thirty-three people who received placebo. These results are also statistically inconclusive.
Simply put, the very best scientific study design currently available to mankind was not used to answer the most important outcomes, and the randomized trials do not support the widely held contention that COVID-19 vaccination using the Pfizer or Moderna brands lowers risk of death. This is unfortunately not the first time that FDA has approved a product based on a less important surrogate end-point rather than the key outcomes of interest.
There are several additional points to consider.
First, the studies’ findings were limited by the fact that their design did not take into consideration previous infection leading to subsequent immunity from COVID-19 infection. While the Covid recovered with prior natural immunity are mandated to be vaccinated in many places, the two randomized trials did not evaluate if the vaccines provide any benefit to them even in terms of the surrogate end-point.
Second, because both trials mostly excluded groups at highest risk of dying from COVID-19, such as the frail elderly and the very obese, we cannot know from the trials how well the vaccines protect these populations.
Thirdly, most study participants were working-age adults, and the very low rates of death from COVID-19 observed in both studies should serve to remind us of how minimal the risk is in this age group.
Finally, the two randomized controlled trials did not evaluate the ability of the vaccine to reduce transmission.
Hence, for the most important questions that are related to vaccine mandates, we are forced to rely on observational studies rather than the usual randomized studies that usually form the basis of FDA drug and vaccine approvals.
Based on clinical judgment and observational studies showing that the vaccines reduce Covid mortality, it is reasonable to assume that for older patients, the vaccine’s benefits outweigh its risks and so advocate for their use, though we cannot be absolutely certain they offer protection against death because of the lack of randomized controlled evidence in this age-group and for this outcome. The lack of such evidence is a failure both by the pharmaceutical companies and the FDA, and it is partly to blame for vaccine hesitancy.
A key takeaway message is that absolutist, rigid COVID-19 vaccine mandates such as that put forth by OSHA are not based on the best possible science. Such population-wide mandates run counter to the universal medical dictum of risk stratification, whereby treatment is tailored to individuals based on individual risks and benefits to be accrued. They also violate the dominant philosophy of evidence-based medicine, which requires the use of the best possible study designs as evidence when making decisions about patient care.
Vaccine mandates, which are enormously costly and terribly divisive, are a cure worse than the disease, and without solid evidence, they are likely to increase vaccine hesitancy rather than vaccine confidence — not just for the Covid vaccines, but also for existing lifesaving vaccines against, for example, measles and polio.
Allon Friedman is a Professor of Medicine at Indiana University School of Medicine and a medical researcher focusing on topics related to kidney disease. The ideas expressed in the article are entirely his own and not necessarily those of his employer.