Mixture separability loss in a deep convolutional network for image classification

• Published in: IET image processing Vol. 13; no. 1; pp. 135 - 141
• Main Authors: Do, Trung Dung, Jin, Cheng-Bin, Nguyen, Van Huan, Kim, Hakil
• Format: Journal Article
• Language: English
• Published: The Institution of Engineering and Technology 01.01.2019
• Subjects: applying cross‐entropy loss; between‐class loss; Canadian institute for advanced research; computer vision; convolutional layers; deep convolutional network; entropy; feedforward neural nets; image classification; image representation; inter‐class images; intra‐class images; learning (artificial intelligence); loss function; machine learning; mixture separability loss; MSL; network structures; novel cost function; Research Article; self‐collected Inha computer vision lab gender dataset; street view house number; training images; well‐known networks; within‐class loss; inter-class images; within-class loss; deep convolutional network; well-known networks; image classification; street view house number; self-collected Inha computer vision lab gender dataset; MSL; training images; novel cost function; machine learning; loss function; entropy; intra-class images; between-class loss; image representation; computer vision; mixture separability loss; Canadian institute for advanced research; learning (artificial intelligence); convolutional layers; network structures; feedforward neural nets; applying cross-entropy loss
• ISSN: 1751-9659; 1751-9667
• DOI: 10.1049/iet-ipr.2018.5613

In machine learning, the cost function is crucial because it measures how good or bad a system is. In image classification, well-known networks only consider modifying the network structures and applying cross-entropy loss at the end of the network. However, using only cross-entropy loss causes a network to stop updating weights when all training images are correctly classified. This is the problem of early saturation. This study proposes a novel cost function, called mixture separability loss (MSL), which updates the weights of the network even when most of the training images are accurately predicted. MSL consists of between-class and within-class loss. Between-class loss maximises the differences between inter-class images, whereas within-class loss minimises the similarities between intra-class images. They designed the proposed loss function to attach to different convolutional layers in the network in order to utilise intermediate feature maps. Experiments show that a network with MSL deepens the learning process and obtains promising results with some public datasets, such as Street View House Number, Canadian Institute for Advanced Research, and the authors’ self-collected Inha Computer Vision Lab gender dataset.