February 13, 2023
I’m Dr. Diana Zuckerman, president of the National Center for Health Research. Our nonprofit research center scrutinizes the safety and effectiveness of medical products, and we don’t accept funding from companies that make those products. So, I have no conflicts of interest, other than being a Medicare beneficiary.
My perspective is based on my current position as well as my post-doc training in epidemiology and public health, my previous policy positions at Congressional Committees with oversight over the U.S. Department of Health and Human Services, my previous position as the Director of Policy, Planning and Legislation at an HHS agency, and as a previous faculty member and researcher at Harvard and Yale. Perhaps most important, I was privileged to serve as a member of MEDCAC for two terms, so I am very familiar with your important work.
When I served on MEDCAC, I was impressed with the generally high quality of the evidence that was considered, but that evidence often had one fatal flaw: The studies frequently focused on White patients under the age of 65, with few if any patients over 70. As is often the case, the research focused on the youngest, healthiest sick patients, in order to reduce the confounding impact of co-morbidities. But as any Medicare beneficiary can tell you, most of us have at least some co-morbidities. For that reason, evidence needs to be focused on representative patients. And the numbers of those patients needs to be large enough to conduct subgroup analyses to determine if the benefits outweigh the risks for those types of patients.
AHRQ and Hopkins did a great job and I generally support their proposed requirements. There are just a few that I think are especially essential and in some cases the wording should be more precise:
CONTEXT
“Study results …provide evidence of net benefit. A product can have benefits but what’s essential is that the benefits outweigh the risks.
I also agree that “It is essential that the evidentiary threshold for the primary outcome(s) demonstrate clinically meaningful differences with sufficient precision.”
OUTCOME is also closely related to #5: The primary outcome(s) for the study are clinically meaningful and important to patients. That’s essential “A surrogate outcome that reliably predicts these outcomes may be appropriate for some questions” – the emphasis should be on “reliably predicts”
POPULATION includes an essential new requirement: The study population should “reflect the demographic and clinical diversity among the Medicare beneficiaries who are the intended users of the intervention.” This includes racial and ethnic backgrounds, gender, and socio-economic status, at a minimum.
GENERALIZABLE
This new recommendation that beneficiaries should be studied “in their usual sites of care” is very important. But it is weakened with the words: “When feasible and appropriate for answering the CED question, data for the study should come from beneficiaries in their usual sites of care…” It is always appropriate and it is essential that it be feasible
DATA QUALITY – this could be worded more clearly
The data should be complete, accurate, of sufficient duration of observation and of sufficient sample size.
DESIGN-SUBPOPULATIONS
This makes it clear that it is not sufficient to have diversity – it is essential to analyze demographic subpopulations, defined by gender and age, as well as clinically- relevant subgroups as motivated by existing evidence.
DATA SHARING is important –of course!
In summary, having statistically significant results is necessary but not sufficient. Studying patients who are diverse in terms of race, ethnicity, gender and age is necessary but not sufficient. The data generated must be relevant to Medicare beneficiaries, must be valid and reliable, and the results must be clinically meaningful. Medicare beneficiaries have gotten older, and so the studies need to include and analyze those older patients, for whom the benefits might be smaller and the risks might be greater. We all understand the pressure to be flexible, but there is no doubt that exceptions can become huge loopholes that are harmful to Medicare and to patients. For example, there are surrogate endpoints that reliably predict clinical outcomes, but not all do, and some only reliably predict outcomes for a subgroup of patients that might not be typical of Medicare beneficiaries.
Thank you for the opportunity to speak today.