January 24, 2022
National Center for Health Research’s Comments on FDA’s Draft Guidance for Industry on Assessing Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making for Drug and Biological Products
We are writing to express our views on the Food and Drug Administration’s draft guidance for industry regarding Real-World Data: Assessing Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making for Drug and Biological Products.
The National Center for Health Research (NCHR) is a nonprofit think tank that conducts, analyzes, and scrutinizes research on a range of health issues, with particular focus on which prevention strategies and treatments are most effective for which patients and consumers. We do not accept funding from companies that make products that are the subject of our work, so we have no conflicts of interest.
The FDA’s draft guidance provides detailed information and recommendations for sponsors on the selection of data sources, the development and validation of definitions for study design elements and data provenance and quality during data accrual and curation when using real-word data (RWD) and real-world evidence (RWE).
FDA provides several important specific examples of necessary steps for the use of RWD and RWE. For example, sponsors will be required to predefine essential elements of the study design (including the analyses), as well as how each element was ascertained from the source. The FDA draft guidance addresses the reliability and relevance of the data, by carefully assessing the appropriateness and limitations of different data sources. Some of those limitations highlighted by FDA include: the socio-economic makeup of patient groups across different payment programs may substantially differ; medical claims data may not adequately specify a particular disease; and that the data generated for clinical use may differ across different healthcare systems depending on variations in clinical practices.
However, we have additional concerns regarding limitations, and we urge that these limitations be added to the final guidance.
1) RWE cannot always provide meaningful results that would help patients and physicians make informed treatment choices. For example, RWE will usually not explain why a patient changed a medication, which may be more important than the percentage of patients whose medication was changed.
2) Outcomes that affect quality of life are often not included in RWD. For example, pain, nausea, or a patient’s ability to walk up a flight of stairs without feeling winded will not be coded in medical records or billing data unless it is treated with a prescription drug or other paid intervention. Although not life-threatening, these adverse events and side effects can have a dramatically debilitating impact on patients. The lack of that information is a serious limitation when RWE is used to support FDA approval. These shortcomings in evaluating and recording quality of life variables should be highlighted in the guidance. It is crucial to consider patients’ quality of life and not just their prescription drugs or medical interventions when relying on RWE. Unfortunately, in many cases the types of complications, adverse events, and other quality of life information will not be available in RWE data sources.
3) We urge the FDA to focus more explicitly on potential biases in the draft guidance. For example, relying on RWD from certain types of devices, such as data recorded from wearable medical devices or fitness apps, may create a bias towards younger, more technology literate patients, who may be healthier than other population groups.
Although well-designed RWD and RWE studies can provide useful information about drug safety and even effectiveness, such as through their large, diverse samples, the lack of information about patients’ quality of life, the reasons why they discontinued or changed their treatment, and other key issues are major shortcomings if RWD and RWE studies replace rather than supplement data from well-designed clinical trials. We therefore strongly encourage the FDA to make it clear that studies based on RWE should only be considered to provide supplemental evidence that can support regulatory decision-making for drugs and biological products. The FDA needs to clearly inform sponsors that RWE studies cannot replace traditional, randomized, blinded, controlled clinical trials. The final guidance should make it clear that RWE should not be used as primary evidence for regulatory decisions.
The National Center for Health Research can be reached at email@example.com or at (202) 223-4000.