August 21, 2023
The National Center for Health Research (NCHR) appreciates the opportunity to express our views regarding the Centers for Medicare and Medicaid Services (CMS) National Coverage Analysis Evidence Review Proposed Guidance.
NCHR is a nonprofit think tank that conducts, analyzes, and scrutinizes research on a range of health issues, with a particular focus on which prevention strategies and treatments are most effective for which patients and consumers. We do not accept funding from companies that make products that are the subject of our work, so we have no conflicts of interest.
According to CMS, the Proposed National Coverage Analysis (NCA) Guidance is intended to “provide a framework for more predictable and transparent evidence development.” We appreciate CMS providing additional information regarding their standards for conducting NCAs to assess whether evidence is sufficient to support a finding that a product or service meets the “reasonable and necessary” coverage standard as part of the NCD development process.
We support the methodological attributes of clinical studies that are described in the guidance. Specifically, the “use of randomization”, “large sample sizes”, and “blinding” are essential methods to determine solid scientific evidence and reduce bias. In addition, we appreciate that CMS carefully considers both the funding source and potential conflicts of interest for study investigators when evaluating the methodological rigor of a study.
The proposed NCA guidance notes that in making NCDs, “CMS considers the totality of the evidence across multiple dimensions, including study design and conduct.” CMS focuses on benefits that generally are “clinically meaningful and durable rather than marginal or short-lived.” CMS states that it typically places greater emphasis on health outcomes that are important to patients and their caregivers. These include quality of life, functional status, duration of disability, morbidity, and mortality, while placing “less emphasis on outcomes in which patients often have a less direct interest, such as intermediate outcomes, surrogate outcomes, and laboratory or radiographic responses.” We strongly agree with CMS’s priorities that emphasize evidence of meaningful health outcomes, particularly morbidity and mortality, since those can be evaluated more objectively.
This guidance also discusses the importance of the generalizability of findings from relevant studies to the Medicare beneficiary population. We agree that in order to interpret and generalize conclusions to the Medicare beneficiary population, studies “may need to match or stratify their intervention and control groups by patient age, co-morbidities, disabling conditions, etc.” Later in the guidance these criteria are expanded to include sex, race/ethnicity, and the severity of the disease, which is essential; unfortunately, too many studies have results that are not generalizable to Medicare beneficiaries because of the lack of people of color and patients under the age of 65, often with few if any patients over 70 or 75. As is often the case, research typically focuses on the youngest, healthiest sick patients, in order to reduce the confounding impact of co-morbidities and maximize the chances that the results will indicate benefits rather than harm. To ensure that medical interventions will benefit Medicare beneficiaries, data need to include enough patients to conduct subgroup analyses to determine if the benefits outweigh the risks for those types of patients.
Finally, regarding the strength of evidence assessments, we agree that CMS-approved CED studies may generally be more persuasive than observational studies because the study design, analysis plan, and data sources are generally prespecified and posted on ClinicalTrials.gov, to ensure that the study design, implementation, and analyses are not changed post hoc. Registration with ClinicalTrials.gov helps to ensure that Medicare beneficiaries and healthcare professionals are aware of studies that are underway, will soon be completed, and are supposed to already be completed; this will facilitate better informed decision making. Information about the study’s pre-specified primary endpoints and analyses help ensure transparency about how the study was meant to be implemented. It is unfortunate that the results of clinical trials are often not reported in a timely or comprehensive manner on ClinicalTrials.gov, because study results are not always published in medical journals. Moreover, studies that are reported in press releases and journals sometimes deviate from the study design and planned analyses, which raises questions about the integrity of the data. We urge CMS to continue to promote transparency in the studies that formulate their NCD and CED determinations.