Quaternion-Valued Echo State Networks

• Published in: IEEE transaction on neural networks and learning systems Vol. 26; no. 4; pp. 663 - 673
• Main Authors: Yili Xia, Jahanchahi, Cyrus, Mandic, Danilo P.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.04.2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Augmented quaternion statistics; Circularity; Computational modeling; Computer Simulation; Covariance matrices; echo state networks (ESNs); Humans; Learning systems; Linear Models; Mathematical analysis; Networks; Neural networks; Neural Networks (Computer); Neurons; Nonlinear Dynamics; Nonlinearity; Quaternions; Reservoirs; second-order noncircularity; Three dimensional; Vectors; widely linear model; Augmented quaternion statistics; widely linear model; echo state networks (ESNs); second-order noncircularity
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2014.2320715

Quaternion-valued echo state networks (QESNs) are introduced to cater for 3-D and 4-D processes, such as those observed in the context of renewable energy (3-D wind modeling) and human centered computing (3-D inertial body sensors). The introduction of QESNs is made possible by the recent emergence of quaternion nonlinear activation functions with local analytic properties, required by nonlinear gradient descent training algorithms. To make QENSs second-order optimal for the generality of quaternion signals (both circular and noncircular), we employ augmented quaternion statistics to introduce widely linear QESNs. To that end, the standard widely linear model is modified so as to suit the properties of dynamical reservoir, typically realized by recurrent neural networks. This allows for a full exploitation of second-order information in the data, contained both in the covariance and pseudocovariances, and a rigorous account of second-order noncircularity (improperness), and the corresponding power mismatch and coupling between the data components. Simulations in the prediction setting on both benchmark circular and noncircular signals and on noncircular real-world 3-D body motion data support the analysis.