On (in)validating environmental models. 1. Principles for formulating a Turing‐like Test for determining when a model is fit‐for purpose

• Published in: Hydrological processes Vol. 36; no. 10
• Main Authors: Beven, Keith, Lane, Stuart
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.10.2022; Wiley Subscription Services, Inc
• Subjects: Communication; Computation; Decision analysis; Digital twins; Environment models; Environmental modeling; epistemic uncertainties; Epistemology; hydraulic models; hydrologic model; hydrology; hypothesis testing; limits of acceptability; Logic; model validation; Modelling; Principles; Simulators; Uncertainty; Watersheds
• ISSN: 0885-6087; 1099-1085
• DOI: 10.1002/hyp.14704

Model invalidation is a good thing. It means that we are forced to reconsider either model structures or the available data more closely, that is to challenge our fundamental understanding of the problem at hand. It is not easy, however, to decide when a model should be invalidated, when we expect that the sources of uncertainty in environmental modelling will often be epistemic rather than simply aleatory in nature. In particular, epistemic errors in model inputs may well exert a very strong control over how accurate we might expect model predictions to be when compared against evaluation data that might also be subject to epistemic uncertainties. We suggest that both modellers and referees should treat model validation as a form of Turing‐like Test, whilst being more explicit about how the uncertainties in observed data and their impacts are assessed. Eight principles in formulating such tests are presented. Being explicit about the decisions made in framing an analysis is one important way to facilitate communication with users of model outputs, especially when it is intended to use a model simulator as a ‘model of everywhere’ or ‘digital twin’ of a catchment system. An example application of the concepts is provided in Part 2. We suggest that model invalidation is a good thing in that we are then forced to do better. Eight principles for a Turing‐like Test for the evaluation of environmental models are proposed. This allows for a proper consideration of epistemic uncertainties in model evaluation.