March 09, 2022
National Center for Health Research’s Comments on Agency for Healthcare Research and Quality’s Draft Comparative Effectiveness Review on Effectiveness of Telehealth for Women’s Preventive Services
We are writing to express our views on the Agency for Healthcare Research and Quality’s (AHRQ) draft competitive effectiveness review on the Effectiveness of Telehealth for Women’s Preventive Services.
The National Center for Health Research (NCHR) is a nonprofit think tank that conducts, analyzes, and scrutinizes research on a range of health issues, with particular focus on which prevention strategies and treatments are most effective for which patients and consumers. We do not accept funding from companies that make products that are the subject of our work, so we have no conflicts of interest.
We support the objective of this report and agree that there is a need to evaluate the effectiveness, use, and patient preferences regarding telehealth for women’s reproductive healthcare services and intimate partner violence (IPV) services. However, we agree with the report’s assessment that the “systematic review demonstrates a paucity of data to inform the effectiveness,” and there are several limitations in the data that are particularly problematic:
The data on the use of telehealth for reproductive health services has numerous shortcomings because the studies have different interventions that are not comparable. For example, in one survey cited in the report, 93% of telehealth visits took place over the phone, whereas only 7% of them took place on a video call. If both video and phone telehealth meetings are combined, the data can’t really be considered an evaluation of either. The largest RCT cited by the report examined those who received in-clinic services plus “phone-enhanced interventions” of weekly phone calls from a counselor until they started taking oral contraceptives, which was followed by 6 months of monthly counseling by phone. This study compared phone-enhanced interventions on top of in-clinic care to those receiving in-clinic care; it did not directly compare telehealth-only visits to in-clinic visits. Therefore, the largest study cited in support of the claim that those receiving telehealth did not differ in contraceptive use at 12 months from those receiving in-clinic care did not actually compare the two. A direct comparison would be needed to determine the comparative benefits of telehealth care to those receiving in-clinic care.
The report also aimed to assess patient preferences regarding telehealth services for reproductive health services, and it noted that half of patients surveyed said that they preferred telehealth services over in-person services. However, the data used to support that claim were collected from April to June 2020. This was during the height of the COVID-19 pandemic shut-downs, before vaccines were available. It is not surprising that many patients preferred telehealth services when many likely deemed in-person services unsafe to attend. The data can’t be considered generalizable to other time periods because the nature of the pandemic and people’s attitudes have changed over time. In order to have a more accurate assessment of overall patient preferences, additional data are needed now that vaccines are available, the numbers of hospitalizations and deaths are decreased, and masks are not required for all in-person visits. In addition, longer-term data are needed regarding patient preferences in order to collect data that are more generalizable to a future where COVID is considered endemic rather than a pandemic.
There are also limitations to the data on telehealth for IPV services. For example, many of the studies cited on IPV services were specifically studying women receiving treatment for substance use disorder. In one study, all participants were pregnant women who were patients at an academic health center. Results from these non-representative study samples cannot be generalized to all women. Other study limitations include the inconsistencies in study design and outcome measures, and the modest evidence that telehealth interventions for IPV are as effective as in-person interventions. Given these limitations and the fact that there is insufficient evidence comparing the benefits of telehealth interventions to in-person screening for IPV, we agree with the recommendations of a number of organizations (such as the World Health Organization) that telehealth interventions should not replace traditional screening for IPV, but rather should be used to augment traditional screening services.
Some differences have been observed regarding demographic differences in the utilization of telehealth services, while some remain to be explored. There are a variety of demographics that need to be assessed, such as geographic locations, ethnicity, age, and socioeconomic status. Understanding the reasons for any observed differences is necessary to assess how different groups benefit from or are disadvantaged by telehealth services. Until that is determined, it will be difficult to reduce disparities in usage and possibly in benefits.
Overall, the limitations of the data cited undermine the credibility of the report’s claims about effectiveness, use, and patient preferences. A major shortcoming is the low number of RCTs that compare telehealth to in-person services. While the report notes the gaps in the evidence and that more research is needed, it is not explicit enough about the shortcomings of the data available. The report needs to discuss the data limitations more explicitly in all sections.
The National Center for Health Research can be reached at info@center4research.org or at (202) 223-4000.