Two-Timescale Multilayer Recurrent Neural Networks for Nonlinear Programming

• Published in: IEEE transaction on neural networks and learning systems Vol. 33; no. 1; pp. 37 - 47
• Main Authors: Wang, Jiasen, Wang, Jun
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Global optimization; Mathematical model; Multilayers; Neural networks; Neurodynamic optimization; Neurodynamics; Nonhomogeneous media; Nonlinear programming; Programming; Quadratic programming; Recurrent neural networks; Time; two-timescale system
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2020.3027471

This article presents a neurodynamic approach to nonlinear programming. Motivated by the idea of sequential quadratic programming, a class of two-timescale multilayer recurrent neural networks is presented with neuronal dynamics in their output layer operating at a bigger timescale than in their hidden layers. In the two-timescale multilayer recurrent neural networks, the transient states in the hidden layer(s) undergo faster dynamics than those in the output layer. Sufficient conditions are derived on the convergence of the two-timescale multilayer recurrent neural networks to local optima of nonlinear programming problems. Simulation results of collaborative neurodynamic optimization based on the two-timescale neurodynamic approach on global optimization problems with nonconvex objective functions or constraints are discussed to substantiate the efficacy of the two-timescale neurodynamic approach.