Stephanie Fox-Rawlings, National Center for Health Research: June 14, 2018
Thank you for the opportunity to speak today on behalf of the National Center for Health Research. I am Dr. Stephanie Fox-Rawlings. Our Center analyzes scientific and medical data to provide objective health information to patients, health professionals, and policy makers. We do not accept funding from drug and medical device companies, so I have no conflicts of interest.
As you’ve heard today, patients need more treatment options for emphysema. However, new products need to provide more than hope. They need to provide real benefit without causing undue risk of harm.
For a device to be worthy of FDA approval, the sponsor needs to demonstrate that its benefits outweigh its risks for most patients. However, the RENEW pivotal trial does not provide evidence that the device is either effective or safe.
The pivotal trial does not provide sufficient evidence that ELEVAIR Endobronchial Coil System improves quality of life, lung function, or exercise capacity.
I realize that using the modified statistical plan, patients treated with the coil did statistically better than the controls for the primary endpoint, the 6-minute walk test. However, the difference was small – only 14.6 meters – which isn’t much considering that the patients were walking over 300 meters. The 6-minute walk test results are influenced by factors such as motivation and number of times patients have performed the test before. In studies testing the reliability of the walk test on patients with COPD, patients typically improved their distance on their second walk test by more than the difference between the control and treatment group. In other words, the improvement could just be “noise” given the other variables that can affect the 6-minute walk test by more than 15 meters. In addition, since patients were not blinded, the experimental group were probably more motivated to walk faster.
The fact that patients in the crossover study did not have the same improvements as patients in the pivotal trial raises more questions about whether the coil improves patients’ ability to perform on the 6 minute walk test.
There are similar concerns about whether the change in FEV1 is clinically meaningful, and how the fact that the patients knew what treatment they were getting affected their quality of life measurement. It is important to note that these tests are surrogate endpoints which do not completely reflect the clinical meaningfulness of the treatment.
Even if we give the pivotal trial results the benefit of these very reasonable doubts regarding effectiveness, these improvements were primarily due to patients outside of the U.S. The differences between the control and treatment arms in the U.S. were much smaller than for those outside the U.S. This could be due to differences in health and BMI in the different countries or it could be due to variations in medical practice or culture. In any event, the FDA should not approve a device that does not work for people in the U.S.
The sponsors continued to follow patients treated with the coil for several years after the study, which could provide valuable data. However, the control group was only followed for the first year. Without a comparison group, it is not possible to interpret these longer term data.
The pivotal trial showed that there were major safety concerns with the coil, as well. There were twice as many adverse events reported for patients treated with the coil compared to controls. ALL treated patients experienced adverse events compared to 88% of controls. Even more concerning, 62% of treated patients experienced serious adverse event(s) compared to 34% of controls, including higher rates of COPD exacerbation, lower respiratory tract infections, respiratory failure, and collapsed lung. Treated patients were hospitalized more often than controls.
It is important to note that 46% of treated patients experienced a serious adverse event that was possibly or probably related to the treatment. There was a similar number of deaths in both treated and control patients, but investigators reported that most deaths in the treatment group were possibly or probably related to the treatment.
Another issue was that 10% of the device implantation procedures included a device malfunction due to “unusually tortuous airway anatomy.” This rate would likely increase if patients that are less carefully selected for the procedure.
Lack of diversity is also a big problem. Over 95% of the study participants were white. Black patients tend to develop COPD at a younger age and with less smoking experience than white patients. The lack of racial diversity of the trial means we can’t assume that non-white patients would benefit from the device at all.
In summary, the pivotal trial does not provide sufficient evidence for effectiveness or safety. Questions about whether the device is effective are exacerbated by the lack of consistency with the crossover data, the differences between US patients and those outside the US, and the lack of blinding. The long-term follow up data are difficult to interpret without controls.
Most important, there are very serious concerns over the safety of the device. A single clinical trial with questionable evidence for effectiveness but with clear evidence of harm does not demonstrate that the benefits outweigh the risks.
The Anesthesiology and Respiratory Therapy Devices Panel of the Medical Devices Advisory Committee voted 3 for approval, while 8 voted against and 1 abstained.