Comprehensive Review of Neural Network-Based Prediction Intervals and New Advances

• Published in: IEEE transactions on neural networks Vol. 22; no. 9; pp. 1341 - 1356
• Main Authors: Khosravi, A., Nahavandi, S., Creighton, D., Atiya, A. F.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.09.2011; Institute of Electrical and Electronics Engineers
• Subjects: Algorithms; Animals; Applied sciences; Artificial neural networks; Bayes Theorem; Bayesian; Bayesian analysis; Bayesian methods; bootstrap; Computation; Computer science; control theory; systems; Computer Simulation; Construction; Cost function; Data processing. List processing. Character string processing; delta; Deltas; Estimation; Exact sciences and technology; Humans; Intervals; Mathematics; mean-variance estimation; Memory organisation. Data processing; Minimization; neural network; Neural networks; Neural Networks (Computer); prediction interval; Predictive Value of Tests; Probability and statistics; Repeatability; Reproducibility; Sciences and techniques of general use; Software; Statistics; Time Factors; Training; Uncertainty; Bayes estimation; High performance; Data analysis; Individual method; Variability; Bayesian; delta; Minimization; Neural network; Statistical forecasting; Mean estimation; mean-variance estimation; Variance; Bootstrapping; Repeatability; Uncertain system; prediction interval; Genetic algorithm; Remolded sample; Bootstrap; Cost function
• ISSN: 1045-9227; 1941-0093
• DOI: 10.1109/TNN.2011.2162110

This paper evaluates the four leading techniques proposed in the literature for construction of prediction intervals (PIs) for neural network point forecasts. The delta, Bayesian, bootstrap, and mean-variance estimation (MVE) methods are reviewed and their performance for generating high-quality PIs is compared. PI-based measures are proposed and applied for the objective and quantitative assessment of each method's performance. A selection of 12 synthetic and real-world case studies is used to examine each method's performance for PI construction. The comparison is performed on the basis of the quality of generated PIs, the repeatability of the results, the computational requirements and the PIs variability with regard to the data uncertainty. The obtained results in this paper indicate that: 1) the delta and Bayesian methods are the best in terms of quality and repeatability, and 2) the MVE and bootstrap methods are the best in terms of low computational load and the width variability of PIs. This paper also introduces the concept of combinations of PIs, and proposes a new method for generating combined PIs using the traditional PIs. Genetic algorithm is applied for adjusting the combiner parameters through minimization of a PI-based cost function subject to two sets of restrictions. It is shown that the quality of PIs produced by the combiners is dramatically better than the quality of PIs obtained from each individual method.