Characterization of a Class of Sigmoid Functions with Applications to Neural Networks

• Published in: Neural networks Vol. 9; no. 5; pp. 819 - 835
• Main Authors: Menon, Anil, Mehrotra, Kishan, Mohan, Chilukuri K., Ranka, Sanjay
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.07.1996; Elsevier Science
• Subjects: Applied sciences; Artificial intelligence; Computer science; control theory; systems; Connectionism. Neural networks; Exact sciences and technology; Hypergeometric series – Legendre equation – Cohen-Grossberg-Hopfield model – Additive model – Fourier transform; Sigmoid functions; Sigmoid functions; Fourier transformation; Additive process; Hypergeometric series; Mathematical method; Neural network; Transfer function; Hopfield model; Legendre polynomial
• ISSN: 0893-6080; 1879-2782
• DOI: 10.1016/0893-6080(95)00107-7

We study two classes of sigmoids: the simple sigmoids, defined to be odd, asymptotically bounded, completely monotone functions in one variable, and the hyperbolic sigmoids, a proper subset of simple sigmoids and a natural generalization of the hyperbolic tangent. We obtain a complete characterization for the inverses of hyperbolic sigmoids using Euler's incomplete beta functions, and describe composition rules that illustrate how such functions may be synthesized from others. These results are applied to two problems. First we show that with respect to simple sigmoids the continuous Cohen-Grossberg-Hopfield model can be reduced to the (associated) Legendre differential equations. Second, we show that the effect of using simple sigmoids as node transfer functions in a one-hidden layer feedforward network with one summing output may be interpreted as representing the output function as a Fourier series sine transform evaluated at the hidden layer node inputs, thus extending and complementing earlier results in this area. Copyright © 1996 Elsevier Science Ltd