Optimized Predictive Coverage by Averaging Time‐Windowed Bayesian Distributions

• Published in: Water resources research Vol. 60; no. 5
• Main Authors: Hsueh, Han‐Fang, Guthke, Anneli, Wöhling, Thomas, Nowak, Wolfgang
• Format: Journal Article
• Language: English
• Published: Washington John Wiley & Sons, Inc 01.05.2024
• Subjects: Bayesian analysis; Bayesian method; Bayesian theory; Calibration; Components; Conditioning; Datasets; data‐driven; Geology; Hydrogeological models; Hydrogeology; Intervals; Mathematical models; Moisture content; Parameters; PDF averaging; Predictions; predictive uncertainty; Probability theory; Soil moisture; Soil moisture models; Statistical analysis; Statistical inference; System dynamics; time series; time‐windowed Bayesian; Uncertainty; Windows (intervals)
• ISSN: 0043-1397; 1944-7973
• DOI: 10.1029/2022WR033280

Hydrogeological models require reliable uncertainty intervals that honestly reflect the total uncertainties of model predictions. The operation of a conventional Bayesian framework only produces realistic (interpretable in the context of the natural system) inference results if the model structure matches the data‐generating process, that is, applying Bayes' theorem implicitly assumes the underlying model to be true. With an imperfect model, we may obtain a too‐narrow‐for‐its‐bias uncertainty interval when conditioning on a long time‐series of calibration data, because the assumption of a quasi‐true model becomes too strict. To overcome the problem of overconfident posteriors, we propose a non‐parametric Bayesian method, called Tau‐averaging method: it applies Bayesian analysis on sliding time windows along the data time series for calibration. Thus, it obtains so‐called transitional posteriors per time window. Then, we average these into a wider predictive posterior. With the proposed routine, we explicitly capture the time‐varying impact of model error on prediction uncertainty. The length of the calibration window is optimized to maximize goal‐oriented statistical skill scores for predictive coverage. Our method loosens the perfect‐model‐assumption by conditioning only on small windows of the data set at a time, that is, it assumes that “the model is sufficient to follow the system dynamics for a smaller duration.” We test our method on two cases of soil moisture modeling and show how it improves predictive coverage as compared to the conventional Bayesian approach. Our findings demonstrate that the proposed method convincingly overcomes the overconfidence drawback of Bayesian inference under model misspecification and long calibration time‐series. Plain Language Summary Mathematical models mimic environmental systems to match what we see, and to predict what will happen. Unfortunately, such models are always simplifications of reality, balancing their complexity between manageability and accuracy. Consequently, interpreting model‐based conclusions requires caution. Assume a model has ten adjustable parameters to make it match with a system. The best‐possible achievable fit to observations is imperfect. Yet, statistical tools indicate we knew these parameters perfectly well after adjustment, especially when adjusting on long data series. Then, we might start believing that this model's adjusted predictions are perfect. We call this “overconfidence.” Ways to overcome overconfidence include extending models by statistical components, making them predict intervals and probabilities rather than exact numbers. However, adjusting these additional statistical components has been difficult to date. In our new approach, we force the model only to match short time windows of the data, and move this window through the whole data set. As we use little data per window, we reduce the overconfidence effect. Instead, the model adjusts parameters and predicts outputs differently in each window. To make predictions, we combine the outputs into a more robust result, such that the testing data fall inside the intervals generated by our method. Key Points We propose a data‐driven Bayesian method to obtain realistic uncertainty estimates despite model errors Our method builds on a statistically rigorous, time‐windowed Bayesian framework without prior assumptions about error sources or patterns The method is confirmed to provide realistic predictive coverage with two synthetic test cases and a real‐world lysimeter case study