National Center for Health Research: August 16, 2018
National Center for Health Research’s Public Comment on General Guiding Principles to Apply Systematic Review in TSCA Risk Evaluations
[Docket ID EPA-HQ-OPPT-2018-0210]
The National Center for Health Research is a nonprofit think tank that conducts, analyzes, and scrutinizes research, policies, and programs on a range of issues related to health. We do not accept funding from companies that make products that are the subject of our work.
We strongly support efforts of the Environmental Protection Agency (EPA) to improve chemical review, as required by the 2016 Frank H. Lautenberg Chemical Safety for the 21st Century Act. EPA’s draft guidance for applying a systematic review process for risk evaluations conducted under the Toxic Substances Control Act will have a large impact on how and whether risk evaluations can meet the obligations under this law. It will have an impact on whether EPA can make decisions based on the “best available science” and the “weight of the scientific evidence.” It will also affect whether EPA fulfills its mission to protect human health and the environment. Moreover, it will affect whether the EPA’s risk evaluations will identify sources of exposure to the chemicals under evaluation and harms so that steps can be taken to minimize risk or whether it allows continued unnecessary exposure and harm.
EPA’s draft guidance should be consistent with the process advocated by scientific experts, such as those at the Toxicology Program, and the National Academies of Sciences, Engineering, and Medicine. EPA should follow well-established, reliable, and scientifically validated methods for conducting systematic review. If the EPA chooses alternative methods, then the process should undergo very comprehensive intra-agency, interagency and external peer review to determine if the methods are reliable and validated before being implemented.
EPA should follow well-established, reliable, and scientifically validated methods for conducting systematic review. If the EPA chooses alternative methods then the process should undergo intra-agency, interagency and external peer review to determine if the methods are reliable and validated before being implemented.
The systematic review process for risk evaluations should be designed to consider and assess all available evidence to ensure that decisions can be based on the totality of the evidences. Assessment of evidence should be based on a holistic view of study quality, including the validity and reliability of the methods and results, and the relevance of the results.
It would be inappropriate to categorize the quality of a study or other source based primarily on the information provided about the methods and results. The amount of information provided differs due to various factors, including publication source, regulatory requirements, and year. Therefore, relying solely on a few requirements concerning this information will inevitably inappropriately classify high quality, useful data.
This is particularly problematic if EPA uses industry standards for reporting, such as requiring compliance with Good Laboratory Practice standards. These standards are used only to fulfill regulatory requirements and are not relevant to categorizing the quality of a scientific study. These standards are not applicable to and therefore rarely used by academic researchers. Relying on such standards would exclude or downgrade many important, well-designed and well-conducted studies that are essential for understanding risk. Unfortunately, those industry reporting requirements do not mean that the study is appropriately designed, conducted appropriately, valid, or reliable. A study that follows Good Laboratory Practice reporting standards can in fact be specifically designed to not identify harms. In contrast, studies that do not use such reporting standards can be scientifically superior. Given the types of research that use industry standards, requiring such standards will introduce bias, reduce transparency, and limit the completeness, accuracy, and usefulness of the analysis.
Determining study quality is only one component of the systematic review process. The draft guidance should discuss how to evaluate and integrate studies to assess the evidence as a whole. This includes acknowledgement of knowledge gaps and recognition of sources of bias. This is crucially important for drawing accurate and meaningful conclusions.
Protecting the public health requires that EPA’s draft guidance for applying a systematic review process for risk evaluations is scientifically valid, reliable, and accepted by the scientific community. It should consider the totality of the scientific evidence without often irrelevant or industry-centric requirements that would reduce the impact of high quality sources or exclude them.
Thank you for the opportunity to comment on the draft guidance for applying systematic review process for risk evaluations.
The National Center for Health Research can be reached through Stephanie Fox-Rawlings, PhD at sfr@center4research.org or at (202) 223-4000.