July 29, 2022.
I am James Castro-Argueta, and I am a medical student completing my degree at the George Washington School of Medicine and Health Sciences. I appreciate the opportunity to speak today on behalf of the National Center for Health Research. Our nonprofit research center conducts research and analyzes scientific data to provide objective health information to patients, health professionals, and policymakers. We do not accept funding from drug, medical device, or tobacco companies, so I have no conflicts of interest.
We do not support the proposed reclassification of the two skin lesion analyzers, MelaFind and Nevisense, from Class III to Class II. We have 2 major concerns:
1) There is no guarantee that newly developed devices cleared through the 510(k) process would be as accurate as those currently on the market. For that reason, newly developed devices should be reviewed through the PMA process.
2) There is a clear risk of false positive and false negative results, misuse, and device failure, and the FDA has not provided adequate evidence that these risks can be adequately mitigated with the Class II special and general controls listed in the executive summary. As described, FDA cannot ensure that there will be reasonable assurance of safety or effectiveness over the lifespan of the newly submitted devices.
For example, the FDA mentions that clinical testing and non-clinical testing need to be performed to determine whether the devices have an acceptable specificity and sensitivity, but does not specify what would be considered “acceptable” nor does the agency specify how often this testing should take place. This is a problem because both of these devices are much more dynamic than was described by FDA’s summary document. Each of these devices will be subject to future software and hardware updates that may change sensitivity and/or specificity. This week’s high-risk recall of Baxter Healthcare’s Abacus Software is a useful reminder that software glitches can seriously harm patients. Where is FDA’s evidence that special controls will keep up with future software updates? It seems to be lacking. As the software undergoes updates that may change the interface or use, it is essential to ensure patient safety, and that can best be done within a PMA process that requires post-market studies. This is especially crucial since these updates could affect the devices’ poorly understood AI/ML algorithm.
Furthermore, the special controls require that adequate information be provided on device labels regarding its operation, intended use, intended users, intended patients, intended lesions, and more. However, in practice, we all recognize that this is only helpful if the label is carefully read. In reality, many healthcare providers take their cues from the device sales reps, which definitely is not sufficient as a “special control”. For that reason, I would suggest instead that the software itself be required to make clear what the intended use is and who should perform the assessment as a physician operates the system. That would help ensure that each operator is routinely reminded of the limitation of the device.
Additionally, the background materials state that an experienced dermatologist who has taken a training program should be operating these devices. However, there is no clarity on who would be considered an experienced dermatologist, what the requirements for the training program are, who is teaching the program, and what are the requirements for passing the training program. We know that online training for physicians is often started but not completed, and even when completed is not necessarily an effective teaching tool.
I also reviewed the studies and was concerned by the lack of patient diversity. One of the studies done on Melafind had a patient population that was 98% white. Other studies were conducted in countries such as the UK, Sweden, and Germany, on populations that may not be representative of the US in terms of race, ethnicity, or other demographic variables that could affect testing. We agree with the referenced Cochrane review article that concluded that new studies are needed to evaluate the use of these types of diagnostic aids in more representative populations.
Bottom line: Class III devices are held to a higher standard than Class II devices, and melanoma is a potentially fatal disease. Down-classifying these devices to Class II would mean no clinical trials or evidence of accuracy would be required for new devices of this type. Although the FDA believes that current devices seem to have an acceptable specificity and sensitivity, there is no guarantee that future devices of this type would be as accurate as the current devices. Inaccurate diagnoses of melanoma either through false positives or false negatives fit the definition of being “high risk” and therefore belong in Class III and not Class II.