A Two-Layer Recurrent Neural Network for Nonsmooth Convex Optimization Problems

• Published in: IEEE transaction on neural networks and learning systems Vol. 26; no. 6; pp. 1149 - 1160
• Main Authors: Qin, Sitian, Xue, Xiaoping
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2015
• Subjects: Algorithms; Biological neural networks; Complexity theory; Computer Simulation; Convex functions; Global convergence; Lyapunov function; Neural Networks, Computer; Nonlinear Dynamics; nonsmooth convex optimization; Optimization; Programming; Recurrent neural networks; two-layer recurrent neural network; nonsmooth convex optimization; two-layer recurrent neural network; Global convergence; Lyapunov function
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2014.2334364

In this paper, a two-layer recurrent neural network is proposed to solve the nonsmooth convex optimization problem subject to convex inequality and linear equality constraints. Compared with existing neural network models, the proposed neural network has a low model complexity and avoids penalty parameters. It is proved that from any initial point, the state of the proposed neural network reaches the equality feasible region in finite time and stays there thereafter. Moreover, the state is unique if the initial point lies in the equality feasible region. The equilibrium point set of the proposed neural network is proved to be equivalent to the Karush-Kuhn-Tucker optimality set of the original optimization problem. It is further proved that the equilibrium point of the proposed neural network is stable in the sense of Lyapunov. Moreover, from any initial point, the state is proved to be convergent to an equilibrium point of the proposed neural network. Finally, as applications, the proposed neural network is used to solve nonlinear convex programming with linear constraints and L 1 -norm minimization problems.