Will Covid-19 vaccines save lives? Current trials aren’t designed to tell us

Dear Editor:

Peter Doshi’s article provides an excellent overview of the shortcomings of the FDA’s proposed standards for Covid-19 vaccines. For months, the public has been told that when a vaccine becomes available, “life will go back to normal.” But that is only true if a vaccine protects most people from the most serious form of Covid-19 for an extended period of time, and especially protects the people most at risk – people over 65, people of color, and people with co-morbidities such as Type 2 diabetes, heart disease, and obesity.

Doshi drew similar conclusions in his oral remarks before the FDA Advisory Committee considering Covid-19 vaccine standards, held remotely on October 22.[1] This was a crucial opportunity to urge the FDA to improve their study requirements, at a time when the FDA is under tremendous pressure to get a Covid vaccine on the market as soon as possible. Doshi pointed out that the primary endpoint for the studies focused on people who tested positive and had at least one symptom, but that those symptoms could be a mild cough or sore throat. While some company protocols called for at least 2 symptoms, they also could be mild. Some point out that Johnson & Johnson’s protocol lists the primary endpoint as “moderate to severe/critical disease” [2], which sounds more stringent, but they define the term as moderate “or” severe, so it is actually a distinction without a difference. As Doshi’s editorial points out, a cough a headache with a positive lab test would suffice.[3] In fact, all the protocols have a primary endpoint that combines outcomes that are at least mild but can also include moderate or severe symptoms, including those requiring hospitalization or causing death.

The secondary endpoints for the vaccine studies focus specifically on “severe” disease, but the FDA defines “severe Covid-19,” as Doshi pointed out at the FDA Advisory Committee meeting, such that it could include an otherwise mild case of Covid-19 with a blood oxygen saturation of less than or equal to 93%, which is a level experienced by thousands of apparently healthy people over 65.[4] Moreover, there is no guarantee the trials will include enough hospitalizations or other truly severe cases to draw any conclusions with statistical certainty.

Several Advisory Committee members expressed similar concerns. For example, Dr. Luigi Notarangelo, a laboratory chief at the National Institute of Allergy and Infectious Diseases, described the efficacy measures as “skewed towards mild disease.” Dr. James Hildreth, the CEO of Meharry Medical College, pointed out that “Since severe disease and death occur primarily among minorities with this virus, if we put a vaccine out there that does not address that issue, it’s just going to perpetuate the perception that exists that [the nonwhite] segment of our population does not matter much in dealing with this challenge.”[5]

In contrast, the chair of the FDA Advisory Committee, Dr. Arnold Monto, defended the FDA standards, stating that “things that prevent infection … typically prevent serious disease.” Similarly, Dr. Philip Krause, deputy director of the FDA’s Office of Vaccines Research and Review, stated at the meeting that “there simply does not exist an example in vaccinology of vaccines that are effective against mild disease that are not more effective against severe disease.”[5]

As Doshi points out, not enough is known about Covid-19 to know whether those statements are accurate. He reminds us that only two placebo controlled trials of the impact of influenza vaccines in the community-dwelling elderly have ever been conducted, and neither was designed to detect any difference in hospital admissions or deaths. And, real world evidence has failed to document that the increased popularity of influenza vaccines in the U.S. has resulted in a decline in mortality. Moreover, even if the trend against mild disease and severe disease trend in the same direction, the magnitude of the effect could be quite different.

The target enrollments for most of the vaccine studies are 30,000 adults each, but it is likely that most of those volunteers will not be exposed to SARS-CoV-2 during the first months of the study and relatively few of those will develop severe disease. An additional point brought up at the meeting is that the FDA was only requiring a MEDIAN of 2 months follow-up after the vaccine was administered. The FDA defended the 2 months as a compromise intended to evaluate short-term adverse events, which usually occur within 2 months, while missing some longer-term risks. However, the 2-month median follow-up also drastically limits information about efficacy, by reducing the number of people in the study who will become infected, as well as failing to provide information about how long immunity generated by the vaccine is likely to last. Krause defended the mild Covid-19 endpoint, saying “the trials may need to be almost 10 times as big” if the goal is to focus on severe disease. [5] However, requiring more than 2 months of follow-up for all participants would also increase the number of severe cases for those already enrolled.

Several panel members, such as Dr. Haley Alman-Gans from Stanford University Medical Center and Archana Chatterjee, Dean of Chicago Medical School, also expressed concern about the low standards to determine safety. [6]

Despite these and other concerns expressed at the FDA’s public meeting, the outcome of the meeting will not necessarily reflect them. When the 9-hour meeting ended, Monto briefly summarized the discussion in a way that seemed to reflect his own views that the FDA standards were appropriate, rather than reflecting the concerns expressed by many panel members. The FDA’s Marion Gruber then added her own brief summary, concurring that in her view the Advisory Committee generally agreed with the FDA standards. That wasn’t even close to my impression, but it will be what the FDA’s written summary will say, and what the agency will quote to support their decisions in the future.

Sincerely,
Diana Zuckerman, PhD

References
[1] “Covid-19 Vaccine Trials” (Public statement at FDA advisory committee (VRBPAC) meeting, Oct 22, 2022; video begins at 5:21:23, and slides)

[2] A Study of Ad26.COV2.S for the Prevention of SARS-CoV-2-Mediated COVID-19 in Adult Participants, Janssen, ClinicalTrials.gov identifier NCT04505722, https://clinicaltrials.gov/ct2/show/record/NCT04505722

[3] Doshi P. Covid-19 vaccine trial protocols released. BMJ 2020; 371:m4058. https://doi.org/10.1136/bmj.m4058

[4] Alejandro Rodrıguez-Molinero, A, Narvaiza, L Ruiz, J. MS Galvez-Barron C. “Normal Respiratory Rate and Peripheral Blood Oxygen Saturation in the Elderly Population”, Journal of the American Geriatrics Society, https://onlinelibrary.wiley.com/doi/pdf/10.1111/jgs.12580

[5] Overley, J. “FDA Advisers Fear Coronavirus Vaccines Won’t Help Enough”, Law360, October 22, 2020
https://www.law360.com/health/articles/1322025/fda-advisers-fear-coronav…

[6] Edney A, Langreth R. “FDA Vaccine Rules Challenged as Weak at Advisory Panel Meeting,” Bloomberg Business News, October 22, 2020. https://www.bloomberg.com/news/articles/2020-10-22/fda-vaccine-rules-cha…