Understanding and overcoming the technical challenges in using in silico predictions in regulatory decisions of complex toxicological endpoints – A pesticide perspective for regulatory toxicologists with a focus on machine learning models

• Published in: Regulatory toxicology and pharmacology Vol. 137; p. 105311
• Main Authors: Burgoon, Lyle D., Kluxen, Felix M., Frericks, Markus
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier Inc 01.01.2023
• Subjects: Complex toxicological endpoints; Computational toxicology; Machine learning; Pesticides; Risk assessment; Computational toxicology; Risk assessment; Pesticides; Machine learning; Complex toxicological endpoints
• ISSN: 0273-2300; 1096-0295; 1096-0295
• DOI: 10.1016/j.yrtph.2022.105311

There are many challenges that must be overcome before in silico toxicity predictions are ripe for regulatory decision-making. Today, mandates in the United States of America and the European Union to avoid animal usage in toxicity testing is driving the need to consider alternative technologies, including Quantitative Structure Activity Relationship (QSAR) models, and read across approaches. However, when adopting new methods, it is critical that both new approach developers as well as regulatory users understand the strengths and challenges with these new approaches. In this paper, we identify potential sources of bias in machine learning methods specific to toxicity predictions, that may impact the overall performance of in silico models. We also discuss ways to mitigate these biases. Based on our experiences, the most prevalent sources of bias include class imbalance (differing numbers of “toxic” vs “nontoxic” compounds), limited numbers of chemicals within a particular chemistry, and biases within the studies that make up the database used for model building, as well as model evaluation biases. While this is already complex for repeated dose toxicity, in reproduction and developmental toxicity a further level of complexity is introduced by the need to evaluate effects on individual animal and litter basis (e.g., a hierarchal structure). We also discuss key considerations developers and regulators need to make when they use machine learning models to predict chemical safety. Our objective is for our paper to serve as a desk reference for model developers and regulators as they evaluate machine learning models and as they make decisions using these models. •Machine learning models in computational toxicology, including QSAR models, may be impacted by many sources of bias.•QSAR models are correlative and not causal in nature.•Black box QSAR models should be avoided as we do not know how they operate or generate predictions.•QSAR models are subject to the same reproducibility concerns in the broader machine learning literature.•Regulators need to assess the quality of the studies used to train QSAR models in order to understand if they can be trusted.