Deep Leaky Single-peaked Triangle Neural Networks

Recently, Deep learning has made a great deal of success in processing images, audios, and natural languages and so on. The activation function is one of the key factors in Deep learning. In this paper, according to characteristics of biological neurons, an improved Leaky Single-Peaked Triangle Line...

Full description

Saved in:
Bibliographic Details
Published inInternational journal of control, automation, and systems Vol. 17; no. 10; pp. 2693 - 2701
Main Authors Shan, Chuan-Hui, Guo, Xi-Rong, Ou, Jun
Format Journal Article
LanguageEnglish
Published Bucheon / Seoul Institute of Control, Robotics and Systems and The Korean Institute of Electrical Engineers 01.10.2019
Springer Nature B.V
제어·로봇·시스템학회
Subjects
Activation
Control
Deep learning
Engineering
Intelligent Control and Applications
Machine learning
Mechatronics
Neural networks
Robotics
제어계측공학
LSPTLU neuron
LSPTLU network
Comparative experiment
leaky single-peaked triangle linear unit (LSPTLU)
Online AccessGet full text

Cover

More Information
Summary:Recently, Deep learning has made a great deal of success in processing images, audios, and natural languages and so on. The activation function is one of the key factors in Deep learning. In this paper, according to characteristics of biological neurons, an improved Leaky Single-Peaked Triangle Linear Unit (LSPTLU) activation function is presented for the right-hand response unbounded of Rectified Linear Unit (ReLU) and Leaky ReLU (LReLU). LSPTLU is more in line with the biological neuron essence and achieves the excellent performance of equivalent or beyond ReLU and LReLU on different datsets, e.g., MNIST, Fashion-MNIST, SVHN, IMAGENET, CALTECH101 and CIFAR10 datasets.
Bibliography:http://link.springer.com/article/10.1007/s12555-018-0796-0
ISSN:1598-6446
2005-4092
DOI:10.1007/s12555-018-0796-0