An accelerated learning algorithm for multilayer perceptron networks

• Published in: IEEE transactions on neural networks Vol. 5; no. 3; pp. 493 - 497
• Main Authors: Parlos, A.G., Fernandez, B., Atiya, A.F., Muthusami, J., Tsai, W.K.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.05.1994; Institute of Electrical and Electronics Engineers
• Subjects: Acceleration; ALGORITHMS; Algorithms for functional approximation; Backpropagation algorithms; Control systems; CONTROL THEORY; Convergence; EDUCATION; Error correction; Exact sciences and technology; Force control; Force feedback; GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; LEARNING; MATHEMATICAL LOGIC 990200 > Mathematics & Computers; Mathematical methods in physics; Multilayer perceptrons; NEURAL NETWORKS; Nonlinear control systems; Nonlinear dynamical systems; NONLINEAR PROBLEMS; Numerical approximation and analysis; Numerical optimization; Numerical simulation, solution of equations; Physics; TRAINING; Backpropagation algorithm; Perceptron; Adaptive algorithm; Neural networks; Fast algorithm; Learning algorithm; Multilayer network
• ISSN: 1045-9227; 1941-0093
• DOI: 10.1109/72.286921

An accelerated learning algorithm (ABP-adaptive back propagation) is proposed for the supervised training of multilayer perceptron networks. The learning algorithm is inspired from the principle of "forced dynamics" for the total error functional. The algorithm updates the weights in the direction of steepest descent, but with a learning rate a specific function of the error and of the error gradient norm. This specific form of this function is chosen such as to accelerate convergence. Furthermore, ABP introduces no additional "tuning" parameters found in variants of the backpropagation algorithm. Simulation results indicate a superior convergence speed for analog problems only, as compared to other competing methods, as well as reduced sensitivity to algorithm step size parameter variations.< >