Analysis of the proficiency of fully connected neural networks in the process of classifying digital images. Benchmark of different classification algorithms on high-level image features from convolutional layers

• Published in: Expert systems with applications Vol. 135; pp. 12 - 38
• Main Authors: Janke, Jonathan, Castelli, Mauro, Popovič, Aleš
• Format: Journal Article
• Language: English
• Published: New York Elsevier Ltd 30.11.2019; Elsevier BV
• Subjects: Algorithms; Artificial neural networks; Benchmarks; Classification; Computer vision; Convolutional neural networks; Digital imaging; Image classification; Image filters; Neural networks; Regression analysis; Support vector machines; Convolutional neural networks; Computer vision; Image classification
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2019.05.058

•Research that investigates CNN architectures for classification task.•Two steps method that combines CNN with other ML classifiers.•Large experimental phase considering different architectures and datasets.•Results used to provide guidelines on the choice of the best classifiers.•CNNs can improve their performance by using the proposed method. Over the course of research on convolutional neural network (CNN) architectures, few modifications have been made to the fully connected layers at the ends of the networks. In image classification, these neural network layers are responsible for creating the final classification results based on the output of the last layer of high-level image filters. Before the breakthrough of CNNs, these image filters were handcrafted, and any classification algorithm could be applied to their output. Because neural networks use gradient descent to learn their weights subject to the classification error, fully connected neural networks are a natural choice for CNNs. But a question arises: Are fully connected layers in a CNN superior to other classification algorithms? In this work, we benchmark different classification algorithms on CNNs by removing the existing fully connected classifiers. Thus, the flattened output from the last convolutional layer is used as the input for multiple benchmark classification algorithms. To ensure the generalisability of the findings, numerous CNNs are trained on CIFAR-10, CIFAR-100, and a subset of ILSVRC-2012 with 100 classes. The experimental results reveal that multiple classification algorithms, namely logistic regression, support vector machines, eXtreme gradient boosting, random forests and K-nearest neighbours, are capable of outperforming fully connected neural networks. Furthermore, the superiority of a particular classification algorithm depends on the underlying CNN structure and the nature of the classification problem. For classification problems with many classes or for CNNs that produce many high-level image features, other classification algorithms are likely to perform better than fully connected neural networks. It follows that it is advisable to benchmark multiple classification algorithms on high-level image features produced from the CNN layers to improve classification performance.