Influence Area of Overlap Singularity in Multilayer Perceptrons

The existing overlap singularities in the parameter space significantly affect the learning dynamics of the multilayer perceptrons. From the obtained theoretical learning trajectories near overlap singularity, when the learning process has been affected by the overlap singularity, the influence area...

Full description

Saved in:
Bibliographic Details
Published inIEEE access Vol. 6; pp. 60214 - 60223
Main Authors Guo, Weili, Yang, Yuan, Zhou, Yingjiang, Tan, Yushun, Wei, Haikun, Song, Aiguo, Pang, Guochen
Format Journal Article
LanguageEnglish
Published Piscataway IEEE 2018
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Analytical models
Automation
dynamics
Error analysis
influence area
information geometry
Learning
Multilayer perceptrons
overlap singularity
Singularities
Training
Trajectory
Online AccessGet full text

Cover

More Information
Summary:The existing overlap singularities in the parameter space significantly affect the learning dynamics of the multilayer perceptrons. From the obtained theoretical learning trajectories near overlap singularity, when the learning process has been affected by the overlap singularity, the influence area of the overlap singularity is just the line space where the two hidden units equal to each other. However, in the practical applications, different case has been observed and the influence area of such singularity may be larger. By analyzing the generalization error of multilayer perceptrons, we find that the error surface is much flatter near overlap singularity and the singularity would have much larger influence area. Finally, the validity of the obtained results are verified by taking an artificial experiment and two real-data experiments.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2018.2873811