Distributed Parallel Support Vector Machines in Strongly Connected Networks

• Published in: IEEE transactions on neural networks Vol. 19; no. 7; pp. 1167 - 1178
• Main Authors: Yumao Lu, Roychowdhury, V., Vandenberghe, L.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.07.2008; Institute of Electrical and Electronics Engineers
• Subjects: Acceleration; Algorithms; Applied sciences; Artificial intelligence; Cascades; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Connectionism. Neural networks; Convergence; Costs; Data mining; Data processing. List processing. Character string processing; distributed data mining; Exact sciences and technology; Iterative algorithms; Memory organisation. Data processing; Network servers; Network topology; Networks; Neural networks; parallel computing; Parallel processing; Servers; Servers (computers); Software; strongly connected networks (SCNs); support vector machine (SVM); Support vector machine classification; Support vector machines; Training; Training data; Parallel algorithm; Data analysis; Statistical analysis; strongly connected networks (SCNs); support vector machine (SVM); Parallel machines; Data transmission; Information extraction; Neural network; Distributed system; Data mining; Synchronization; Distributed computing; Convergence; distributed data mining; Vector support machine; parallel computing
• ISSN: 1045-9227; 1941-0093
• DOI: 10.1109/TNN.2007.2000061

In this paper, we propose a distributed parallel support vector machine (DPSVM) training mechanism in a configurable network environment for distributed data mining. The basic idea is to exchange support vectors among a strongly connected network (SCN) so that multiple servers may work concurrently on distributed data set with limited communication cost and fast training speed. The percentage of servers that can work in parallel and the communication overhead may be adjusted through network configuration. The proposed algorithm further speeds up through online implementation and synchronization. We prove that the global optimal classifier can be achieved iteratively over an SCN. Experiments on a real-world data set show that the computing time scales well with the size of the training data for most networks. Numerical results show that a randomly generated SCN may achieve better performance than the state of the art method, cascade SVM, in terms of total training time.