Design of continuous-time recurrent neural networks with piecewise-linear activation function for generation of prescribed sequences of bipolar vectors

• Published in: Neural networks Vol. 164; pp. 588 - 605
• Main Authors: Takahashi, Norikazu, Yamakawa, Tsuyoshi, Minetoma, Yasuhiro, Nishi, Tetsuo, Migita, Tsuyoshi
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.07.2023
• Subjects: Bipolar vector; Computer Simulation; Limit cycle; Mathematical programming; Neural Networks, Computer; Neurons; Piecewise-linear activation function; Programming, Linear; Recurrent neural network; Sequence; Time Factors; Bipolar vector; Sequence; Recurrent neural network; Limit cycle; Piecewise-linear activation function; Mathematical programming
• ISSN: 0893-6080; 1879-2782
• DOI: 10.1016/j.neunet.2023.05.013

A recurrent neural network (RNN) can generate a sequence of patterns as the temporal evolution of the output vector. This paper focuses on a continuous-time RNN model with a piecewise-linear activation function that has neither external inputs nor hidden neurons, and studies the problem of finding the parameters of the model so that it generates a given sequence of bipolar vectors. First, a sufficient condition for the model to generate the desired sequence is derived, which is expressed as a system of linear inequalities in the parameters. Next, three approaches to finding solutions of the system of linear inequalities are proposed: One is formulated as a convex quadratic programming problem and others are linear programming problems. Then, two types of sequences of bipolar vectors that can be generated by the model are presented. Finally, the case where the model generates a periodic sequence of bipolar vectors is considered, and a sufficient condition for the trajectory of the state vector to converge to a limit cycle is provided.