Direct model of memory properties and the linear reservoir topologies in echo state networks

• Published in: Applied soft computing Vol. 22; pp. 622 - 628
• Main Authors: Ma, Qianli, Chen, Weibiao, Wei, Jia, Yu, Zhiwen
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.09.2014
• Subjects: Echo state networks; Reservoir computing; Reservoir topology; Short-term memory; Reservoir computing; Echo state networks; Short-term memory; Reservoir topology
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2014.04.038

•A direct relationship between the memory of the linear reservoir and its connectivity is established.•The reservoir topology can be constructed according to the given short-term memory.•Some ESN's reservoirs presented in others works, the DSR and DLR, are special solutions of our model.•The proposed model provides an easy way to adjust the reservoir to the edge of chaos. Echo state networks (ESN), with a large number of randomly connected neurons (called “reservoir”) and a simple linear output neuron, are a kind of novel recurrent neural network (RNN). One of the most important properties of ESN is short-term memory (STM), which is indispensable for time varying information processing. However, due to the random connection of neurons in the reservoir, the relationship between the topological structure of the reservoir and STM in ESN is not fully understood. In this paper, we concentrate on ESN with a linear reservoir, which consists of neurons with identity activation functions. We transform the iterative mathematical model of ESN into a direct one. In the model, we establish a direct relationship between the memory capacity of ESN and its connectivity, which can obtain the reservoir topology through STM in ESN. We find that some reservoir topologies proposed by previous papers are the special solutions of our method. Furthermore, the proposed model can provide an effective way to adjust the reservoir to a state near the edge of chaos, where previous studies have shown that the reservoir can achieve maximum computational capabilities.