August 21, 2023
The National Center for Health Research (NCHR) appreciated the opportunity to express our views regarding the Centers for Medicare and Medicaid Services (CMS) Coverage with Evidence Development Proposed Guidance Document.
NCHR is a nonprofit think tank that conducts, analyzes, and scrutinizes research on a range of health issues, with a particular focus on which prevention strategies and treatments are most effective for which patients and consumers. We do not accept funding from companies that make products that are the subject of our work, so we have no conflicts of interest.
We appreciate the changes that were made to the guidance document in response to the February 13-14 Medicare Evidence Development & Coverage Advisory Committee (MEDCAC) public meeting, which was convened to examine the general requirements for clinical studies submitted for CMS coverage under CED. We support the refinements to the proposed new criteria regarding the need for proper control groups and blinding in CED studies. However, we have minor suggestions that would strengthen these changes.
- The updated CED guidance discusses the importance of blinding but also discusses exceptions for “cases where blinding is not possible or not used.” This should be amended to eliminate “or not used.” Blinding should be required in CED clinical studies, with exceptions only when it is not possible to do so.
- The guidance also states that: “In some cases, an identified evidence deficiency may only be addressed through use of a randomized controlled clinical trial (RCT). In other cases, observational studies that employ advanced designs and analytic methods may be sufficient.” We urge this to be amended to recommend the use of RCTs unless it is not possible to do so. Even in the case that an observation study may be “sufficient”, if a RCT is achievable, it should be the chosen study design.
We also wish to emphasize testimony made by NCHR’s president, Dr. Diana Zuckerman, at the February MEDCAC public meeting. We urge you to incorporate her recommendations into the Clinical Study Standards for CED under Section 1862(a)(1)(E) in the final version of the CED guidance.
- Health Outcome
We strongly agree that the primary outcomes for a study must be clinically meaningful and important to patients and their caregivers. We also agree that only “a validated surrogate outcome that reliably predicts these outcomes may be appropriate for some questions.” However, the interpretation of the definition of “reliably predicts” varies considerably. For example, we agree with the many experts who point out that the amount of amyloid beta plaque on the brain does not reliably predict cognitive impairment; however, some people argue that the biomarker is reliable enough. Surrogate outcomes should only replace clinically meaningful outcomes when the predictive value of the surrogate endpoint is consistent and conclusive.
- Care Setting
The recommendation that beneficiaries should be studied “in their usual sites of care” is very important. But it is weakened with the words: “When feasible and appropriate for answering the CED question, data for the study should come from beneficiaries in their usual sites of care…” It is always appropriate and it is essential that it be feasible, so we urge CMS to strengthen that wording.
- Context
This proposed guidance states that study results are expected to provide “evidence sufficient to assess health outcomes.” However, there is no mention of the need for a net benefit, which was included in the standard discussed during the February 2023 MEDCAC meeting. “Evidence sufficient to assess health outcomes” does not specify that the evidence must indicate meaningful benefits, and even if it did specify meaningful benefits, that would not be sufficient because what is essential is that the benefits outweigh the risks. This should be clearly stated. We are also concerned about the deletion of the need to “establish an evidentiary threshold for the primary outcome(s) so as to demonstrate clinically meaningful differences with sufficient precision,” as was included in the February MEDCAC meeting. We strongly urge that these prior requirements be restored because they are essential to ensure the safety of Medicare beneficiaries.
In summary, we agree that blinding is not always possible (for example, if the most appropriate control group does not require surgery or an injection) but when it is possible, it should be required. Too often, wording that is intended to provide reasonable flexibility instead creates huge loopholes that are harmful to Medicare beneficiaries and to the Medicare program as a whole. Similarly, there are surrogate endpoints that reliably predict meaningful clinical outcomes that are important to patients, but many do not; moreover, some reliably predict outcomes only for a subgroup of patients that might not be generalizable to all Medicare beneficiaries. In addition, when evaluating a clinical study for CED, having statistically significant results is necessary but not sufficient. Studying patients who are diverse in terms of race, ethnicity, gender and age is necessary but not sufficient. The data generated must be relevant to Medicare beneficiaries and must be valid and reliable, so that the results are proven to be clinically meaningful.