Generalized Single-Hidden Layer Feedforward Networks for Regression Problems

• Published in: IEEE transaction on neural networks and learning systems Vol. 26; no. 6; pp. 1161 - 1176
• Main Authors: Wang, Ning, Er, Meng Joo, Han, Min
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2015
• Subjects: Approximation capability; Approximation methods; Educational institutions; extreme learning machine (ELM); Feedforward neural networks; generalized single-hidden layer feedforward networks (GSLFN); Joining processes; polynomial output weights; Polynomials; ridge regression; generalized single-hidden layer feedforward networks (GSLFN); Approximation capability; polynomial output weights; extreme learning machine (ELM); ridge regression
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2014.2334366

In this paper, traditional single-hidden layer feedforward network (SLFN) is extended to novel generalized SLFN (GSLFN) by employing polynomial functions of inputs as output weights connecting randomly generated hidden units with corresponding output nodes. The significant contributions of this paper are as follows: 1) a primal GSLFN (P-GSLFN) is implemented using randomly generated hidden nodes and polynomial output weights whereby the regression matrix is augmented by full or partial input variables and only polynomial coefficients are to be estimated; 2) a simplified GSLFN (S-GSLFN) is realized by decomposing the polynomial output weights of the P-GSLFN into randomly generated polynomial nodes and tunable output weights; 3) both P- and S-GSLFN are able to achieve universal approximation if the output weights are tuned by ridge regression estimators; and 4) by virtue of the developed batch and online sequential ridge ELM (BR-ELM and OSR-ELM) learning algorithms, high performance of the proposed GSLFNs in terms of generalization and learning speed is guaranteed. Comprehensive simulation studies and comparisons with standard SLFNs are carried out on real-world regression benchmark data sets. Simulation results demonstrate that the innovative GSLFNs using BR-ELM and OSR-ELM are superior to standard SLFNs in terms of accuracy, training speed, and structure compactness.