Parallel simulation of multilayered neural networks on distributed-memory multiprocessors

• Published in: Microprocessing and microprogramming Vol. 29; no. 3; pp. 185 - 195
• Main Authors: Yoon, Hyunsoo, Nang, Jong H, Maeng, S.R
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.10.1990; North-Holland
• Subjects: Applied sciences; Artificial intelligence; Backpropagation algorithm; Computer science; control theory; systems; Distributed-memory multiprocessor; Exact sciences and technology; Neural network; Parallel processing; Performance analysis; Parallel processing; Backpropagation algorithm; Neural network; Performance analysis; Distributed-memory multiprocessor; Multiprocessor; Backpropagation; Simulation; Multilayer
• ISSN: 0165-6074
• DOI: 10.1016/0165-6074(90)90005-T

In this paper, we present a parallel simulation of a fully connected multilayered neural network using the backpropagation learning algorithm on a distributed-memory multiprocessor system. In our system, the neurons on each layer are partitioned into p disjoint sets and each set is mapped on a processor of a p-processor system. A fully distributed backpropagation algorithm, necessary communication pattern among the processors, and their time/space complexities are investigated. The p-processor speed-up of the backpropagation algorithm over a single processor is also analyzed theoretically which can be used as a basis in determining the most cost-effective or optimal number of processors. The experimental results with a network of Transputers are also presented to demonstrate the usefulness of our system.