Sparse Bayesian Extreme Learning Machine for Multi-classification

• Published in: IEEE transaction on neural networks and learning systems Vol. 25; no. 4; pp. 836 - 843
• Main Authors: JIAHUA LUO, VONG, Chi-Man, WONG, Pak-Kin
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.04.2014; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Applied sciences; Artificial Intelligence; Bayes methods; Bayes Theorem; Bayesian analysis; Bayesian learning; Classification; Computational modeling; Computer science; control theory; systems; Computer Simulation; Connectionism. Neural networks; Couplings; Data processing. List processing. Character string processing; Estimating; Exact sciences and technology; extreme learning machine (ELM); Learning; Learning and adaptive systems; Memory organisation. Data processing; Models, Theoretical; Neural networks; Neurons; Pattern Recognition, Automated - methods; Software; sparsity; Support vector machines; Training; Bayes estimation; Extreme learning machine; Redundancy; Bayesian learning; Minimization; sparsity; Neural network; classification; Generalized inverse; Modeling; Feedforward neural nets; Least squares method; Sparse representation; Probability learning; Pruning(tree); Feedforward; Learning algorithm; Artificial intelligence; extreme learning machine (ELM); Multiple classification
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2013.2281839

Extreme learning machine (ELM) has become a popular topic in machine learning in recent years. ELM is a new kind of single-hidden layer feedforward neural network with an extremely low computational cost. ELM, however, has two evident drawbacks: 1) the output weights solved by Moore-Penrose generalized inverse is a least squares minimization issue, which easily suffers from overfitting and 2) the accuracy of ELM is drastically sensitive to the number of hidden neurons so that a large model is usually generated. This brief presents a sparse Bayesian approach for learning the output weights of ELM in classification. The new model, called Sparse Bayesian ELM (SBELM), can resolve these two drawbacks by estimating the marginal likelihood of network outputs and automatically pruning most of the redundant hidden neurons during learning phase, which results in an accurate and compact model. The proposed SBELM is evaluated on wide types of benchmark classification problems, which verifies that the accuracy of SBELM model is relatively insensitive to the number of hidden neurons; and hence a much more compact model is always produced as compared with other state-of-the-art neural network classifiers.