Stephanie Fox-Rawlings, PhD, National Center for Health Research, March 21, 2019
Thank you for the opportunity to speak today on behalf of the National Center for Health Research. I am Dr. Stephanie Fox-Rawlings, a neuroscientist. Our Center analyzes scientific and medical data to provide objective health information to patients, health professionals, and policy makers. We do not accept funding from drug and medical device companies, so I have no conflicts of interest.
We all agree that there is an unmet need for treatments for patients with Alzheimer’s disease. The question is whether this TMS device [neuroAD] will help fill that need. As a scientist looking at the data I have to conclude that the evidence presented today does not demonstrate that it works.
The lack of a significant difference between the active treatment and sham treatment have been extensively discussed in the FDA materials. It is only with post hoc analyses that the sponsor was able to find a significant difference and even then it was only at a single time point. As those of you familiar with statistics know, if you test enough comparisons, you are very likely to find a difference that is statistically significant – but that actually occurred by chance. Moreover, the ponsor did not use statistical methods to control for these additional comparisons.
The sponsor provides additional studies to support a benefit, but these studies included a small number of patients and/or were open-label. For those that were double-blind and included a sham, it is important to know what the sham treatment was used and the effectiveness of the blinding. Frequently TMS studies with better blinding have smaller differences.
Even with the meta-analyses and the post-hoc groups, the largest improvements in either the ADAS-Cog or CGIC were still very small and do not appear clinically meaningful. On the ADAS-Cog, the largest mean improvement from baseline was about 2 points, which the FDA pointed out equates to remembering 2 additional words during the word recall portion of the test. But recalling two more or two fewer words can vary on two separate days for many different reasons unrelated to TMS treatment. This is especially true with dementia patients, whose cognitive impairment varies from day to day. The FDA’s experts and the scientific literature both suggest that it is not clinically significant.
The sponsor notes that drugs for Alzheimer’s disease have been approved with similar levels of improvement on the ADAS-Cog scale, but these differences were statistically better than placebo and they were statistically better than placebo on a global scale as well.
Since there doesn’t seem to be a large safety risk, should FDA approve TMS for dementia even though it might not work? Not if FDA approval is going to mean anything to doctors and patients. The FDA standard is a likely positive benefit to risk ratio. There is no evidence for a probable benefit based on these studies.
FDA doesn’t consider cost, but it is relevant. Current TMS treatment often costs at least $300 per session. If this is approved, how many millions or even billions of dollars will be spent on a treatment that does not work. Even worse, families that can’t afford treatment will feel guilty, not realizing that there is no clear evidence that the treatment works.
I want to make sure you also realize that your decision will have implications for similar devices. If this device is cleared, similar devices can be cleared through the 510(k) pathway based primarily on their similarity to this device, with little or no clinical evidence.
Perhaps most important, if we want to find an effective treatment for Alzheimer’s disease, research is needed to scientifically conclude what works and for whom. Perhaps scientists would continue to study TMS for Alzheimer’s disease even if this device is on the market, but it will most likely be for new protocols, including different frequencies or types of TMS or training exercises. If we want to know if this device works as described by the sponsor, better research results should be required before approval.
TMS may eventually be a useful treatment for Alzheimer’s disease, but the conditions necessary to make it effective need to be determined before it is approved. If patients receive a treatment that is not demonstrated to work, it should be in the context of a clinical trial that provides free experimental treatment with appropriate informed consent. It should not be in a clinic where patients and their families are paying thousands of dollars for a treatment that they were told is effective, but isn’t.
Thank you.