Determining and improving the fault tolerance of multilayer perceptrons in a pattern-recognition application

• Published in: IEEE transactions on neural networks Vol. 4; no. 5; pp. 788 - 793
• Main Authors: Emmerson, M.D., Damper, R.I.
• Format: Journal Article Conference Proceeding
• Language: English
• Published: New York, NY IEEE 01.09.1993; Institute of Electrical and Electronics Engineers
• Subjects: Applied sciences; Artificial neural networks; Degradation; Electric, optical and optoelectronic circuits; Electronics; Exact sciences and technology; Fault tolerance; Measurement; Multilayer perceptrons; Neural networks; Nonhomogeneous media; Parallel processing; Redundancy; Very large scale integration; Fault tolerance; VLSI circuit; Empirical method; Multiple layer; Performance analysis; Pattern recognition; Neural network; Damage; Singular value decomposition
• ISSN: 1045-9227
• DOI: 10.1109/72.248456

We investigate empirically the performance under damage conditions of single- and multilayer perceptrons (MLP's), with various numbers of hidden units, in a representative pattern-recognition task. While some degree of graceful degradation was observed, the single-layer perceptron was considerably less fault tolerant than any of the multilayer perceptrons, including one with fewer adjustable weights. Our initial hypothesis that fault tolerance would be significantly improved for multilayer nets with larger numbers of hidden units proved incorrect. Indeed, there appeared to be a liability to having excess hidden units. A simple technique (called augmentation) is described, which was successful in translating excess hidden units into improved fault tolerance. Finally, our results were supported by applying singular value decomposition (SVD) analysis to the MLP's internal representations.< >