Chaotic Time Series Prediction Based on a Novel Robust Echo State Network

• Published in: IEEE transaction on neural networks and learning systems Vol. 23; no. 5; pp. 787 - 799
• Main Authors: Li, Decai, Han, Min, Wang, Jun
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.05.2012; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Artificial intelligence; Bayesian analysis; Bayesian methods; Computer science; control theory; systems; Computer Simulation; Connectionism. Neural networks; Echo state network (ESN); Estimating; Exact sciences and technology; Laplace likelihood function; Learning; Learning and adaptive systems; Mathematical model; Mathematical models; Models, Statistical; Networks; Neural networks; Neural Networks (Computer); Nonlinear Dynamics; Optimization; Pattern Recognition, Automated - methods; Reservoirs; robust model; Robustness; Studies; surrogate function; Training; Training data; Chaos; Bayes estimation; Recurrent neural nets; Parameter estimation; Time series; Gaussian distribution; robust model; Neural network; Modeling; Optimization; Outlier; Gaussian process; Supervised learning; Echo state network (ESN); Laplace likelihood function; Probability learning; surrogate function; Likelihood function
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2012.2188414

In this paper, a robust recurrent neural network is presented in a Bayesian framework based on echo state mechanisms. Since the new model is capable of handling outliers in the training data set, it is termed as a robust echo state network (RESN). The RESN inherits the basic idea of ESN learning in a Bayesian framework, but replaces the commonly used Gaussian distribution with a Laplace one, which is more robust to outliers, as the likelihood function of the model output. Moreover, the training of the RESN is facilitated by employing a bound optimization algorithm, based on which, a proper surrogate function is derived and the Laplace likelihood function is approximated by a Gaussian one, while remaining robust to outliers. It leads to an efficient method for estimating model parameters, which can be solved by using a Bayesian evidence procedure in a fully autonomous way. Experimental results show that the proposed method is robust in the presence of outliers and is superior to existing methods.