Efficient Global Optimization of Two-layer ReLU Networks: Quadratic-time Algorithms and Adversarial Training

• Published in: arXiv.org
• Main Authors: Bai, Yatong, Gautam, Tanmay, Sojoudi, Somayeh
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 06.01.2022
• Subjects: Algorithms; Approximation; Artificial neural networks; Back propagation networks; Complexity; Computational geometry; Convergence; Convexity; Global optimization; Mathematical analysis; Optimization; Robustness; Training

The non-convexity of the artificial neural network (ANN) training landscape brings inherent optimization difficulties. While the traditional back-propagation stochastic gradient descent (SGD) algorithm and its variants are effective in certain cases, they can become stuck at spurious local minima an...