Soft-Margin Softmax for Deep Classification

• Published in: Neural Information Processing Vol. 10635; pp. 413 - 421
• Main Authors: Liang, Xuezhi, Wang, Xiaobo, Lei, Zhen, Liao, Shengcai, Li, Stan Z.
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Classification; CNN; L-Softmax; SM-Softmax; Softmax
• ISBN: 3319700952; 9783319700953
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-70096-0_43

In deep classification, the softmax loss (Softmax) is arguably one of the most commonly used components to train deep convolutional neural networks (CNNs). However, such a widely used loss is limited due to its lack of encouraging the discriminability of features. Recently, the large-margin softmax loss (L-Softmax [1]) is proposed to explicitly enhance the feature discrimination, with hard margin and complex forward and backward computation. In this paper, we propose a novel soft-margin softmax (SM-Softmax) loss to improve the discriminative power of features. Specifically, SM-Softamx only modifies the forward of Softmax by introducing a non-negative real number m, without changing the backward. Thus it can not only adjust the desired continuous soft margin but also be easily optimized by the typical stochastic gradient descent (SGD). Experimental results on three benchmark datasets have demonstrated the superiority of our SM-Softmax over the baseline Softmax, the alternative L-Softmax and several state-of-the-art competitors.