Evaluation of the ToxRTool’s ability to rate the reliability of toxicological data for human health hazard assessments

• Published in: Regulatory toxicology and pharmacology Vol. 72; no. 1; pp. 94 - 101
• Main Authors: Segal, D., Makris, S.L., Kraft, A.D., Bale, A.S., Fox, J., Gilbert, M., Bergfelt, D.R., Raffaele, K.C., Blain, R.B., Fedak, K.M., Selgrade, M.K., Crofton, K.M.
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier Inc 01.06.2015
• Subjects: Data quality; Hazardous Substances - toxicity; Health Impact Assessment - methods; Health Impact Assessment - standards; Human health risk assessment; Humans; Reproducibility of Results; Research - standards; Risk of bias; Software; Systematic review; Toxicology - methods; Toxicology - standards; ToxRTool; Systematic review; ToxRTool; Data quality; Risk of bias; Human health risk assessment
• ISSN: 0273-2300; 1096-0295
• DOI: 10.1016/j.yrtph.2015.03.005

•We used ToxRTool to assess toxicology studies for use in health assessments.•The highest quality studies had the most consistent ratings from the scientists.•Reasons for inconsistencies were that some criteria were subjective and/or unclear.•The intended purpose of the study being assessed could affect its rating.•With refinements, ToxRTool could be useful to the health assessment community. Regulatory agencies often utilize results from peer reviewed publications for hazard assessments. A problem in doing so is the lack of well-accepted tools to objectively, efficiently and systematically assess the quality of published toxicological studies. Herein, we evaluated the publicly available software-based ToxRTool (Toxicological data Reliability assessment Tool) for use in human health hazard assessments. The ToxRTool was developed by the European Commission’s Joint Research Center in 2009. It builds on Klimisch categories, a rating system established in 1997, by providing additional criteria and guidance for assessing the reliability of toxicological studies. It also transparently documents the study-selection process. Eight scientists used the ToxRTool to rate the same 20 journal articles on thyroid toxicants. Results were then compared using the Finn coefficient and “AC1” to determine inter-rater consistency. Ratings were most consistent for high-quality journal articles, but less consistent as study quality decreased. Primary reasons for inconsistencies were that some criteria were subjective and some were not clearly described. It was concluded, however, that the ToxRTool has potential and, with refinement, could provide a more objective approach for screening published toxicology studies for use in health risk evaluations, although the ToxRTool ratings are primarily based on study reporting quality.