A probabilistic learning algorithm for robust modeling using neural networks with random weights

• Published in: Information sciences Vol. 313; pp. 62 - 78
• Main Authors: Cao, Feilong, Ye, Hailiang, Wang, Dianhui
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 20.08.2015
• Subjects: Algorithms; Expectation maximization; Laplace distribution; Learning; Mathematical models; Maximization; Neural networks; Neural networks with random weights (NNRWs); Outliers; Outliers (statistics); Probabilistic methods; Probabilistic robust learning; Probability theory; Outliers; Expectation maximization; Laplace distribution; Probabilistic robust learning; Neural networks with random weights (NNRWs)
• ISSN: 0020-0255; 1872-6291
• DOI: 10.1016/j.ins.2015.03.039

Robust modeling approaches have received considerable attention due to its practical value to deal with the presence of outliers in data. This paper proposes a probabilistic robust learning algorithm for neural networks with random weights (NNRWs) to improve the modeling performance. The robust NNRW model is trained by optimizing a hybrid regularization loss function according to the sparsity of outliers and compressive sensing theory. The well-known expectation maximization (EM) algorithm is employed to implement our proposed algorithm under some assumptions on noise distribution. Experimental results on function approximation as well as UCI data sets for regression and classification demonstrate that the proposed algorithm is promising with good potential for real world applications.