December 18, 2023
We appreciate the opportunity to comment on the FDA’s proposed guidance, “Demonstrating Substantial Evidence of Effectiveness Based on One Adequate and Well-Controlled Clinical Investigation and Confirmatory Evidence.”
The National Center for Health Research (NCHR) is a nonprofit think tank that conducts, analyzes, and scrutinizes research on a range of health issues, with a particular focus on which prevention strategies and treatments are most effective for which patients and consumers. We do not accept funding from companies that make products that are the subject of our work, so we have no conflicts of interest.
The FDA’s draft guidance undermines the level of research standards necessary to demonstrate substantial evidence of a drug’s effectiveness and safety. Basing potentially life-saving or life-threatening treatments on only one study should only be considered under the most urgent circumstances with particularly strong evidence. Therefore, the FDA should make it clear that a randomized, double-blinded, clinical trial is the gold standard for demonstrating drug safety and effectiveness. Only in exceptional circumstances when it is not possible to carry out such studies, should alternative study designs be considered. While clinical trial designs are more complex and costlier, they are clearly superior in terms of reducing bias and cofounding and therefore provide the most accurate information about the impact of an intervention on a meaningful outcome measure.
Moreover, the confirmatory evidence that the FDA proposes to help substantiate a drug’s effectiveness heavily relies on gross generalizations that are detrimental to the scientific process. For example, the FDA suggests that the application of clinical evidence from a related indication may be used to confirm a drug’s effectiveness. It is well-documented that while an infection in one part of the body born from a specific pathogen may be effectively treated with Antibiotic A, this does not mean that Antibiotic A will be successful in treating the same pathogen in a different part of the body. For example, a patient who has a device or hardware infected with Staphylococcus aureus may respond best to treatment with Rifampin (an anti-mycobacterial) while a patient infected with Staphylococcus aureus in the blood may respond best to treatment with Nafcillin (a penicillin) or Cefazolin (a cephalosporin), depending on the clinical scenario. All three of these antibiotics can be used to treat the same pathogen in different areas of the body, yet all three represent entirely different drug classes. Generalizations about the efficacy of a drug in different environments is an unsafe practice and should not be included as an option in an FDA guidance. Thus, we strongly recommend that the FDA oppose the use of clinical evidence from related conditions to confirm a novel drug’s effectiveness.
We also strongly discourage the application of evidence from an animal model to confirm a drug’s effectiveness in humans. Animal models have vastly different anatomy, physiology, drug distribution, and drug metabolism compared to humans. While these studies may reasonably serve as a proof-of-concept for an experimental design or guide pre-clinical studies, they have absolutely no role in serving as substantial evidence to confirm a drug’s effectiveness in humans. Even the animals that are most similar to humans are still drastically different and cannot be used to predict the human response to a drug to any substantial degree. Further, the disease state being studied in animal models is often simulated and does not follow the standard pathophysiologic course seen in humans. Thus, any application of animal data to humans would be based on so many assumptions and generalizations that the FDA should not consider this to be substantial evidence to confirm a drug’s effectiveness.
NCHR concludes that this guidance fails to recognize the dangers to the scientific process that arise from such major generalizations and assumptions. The impact of such generalizations and assumptions could be extremely harmful to patients. Patients deserve drugs that are proven to work, not drugs that may work or even drugs that are likely to work based on animal models or other research that is clearly inferior to the gold standard of randomized, double blind clinical trials. Our comment provides several specific areas of improvement, the guidance as a whole should substantially improve by incorporating stronger scientific standards of evidence that is robust and does not rely on assumptions and inferences.