Transfer learning features for predicting aesthetics through a novel hybrid machine learning method

• Published in: Neural computing & applications Vol. 32; no. 10; pp. 5889 - 5900
• Main Authors: Carballal, Adrian, Fernandez-Lozano, Carlos, Heras, Jonathan, Romero, Juan
• Format: Journal Article
• Language: English
• Published: London Springer London 01.05.2020; Springer Nature B.V
• Subjects: Aesthetics; Algorithms; Artificial Intelligence; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Correlation analysis; Data Mining and Knowledge Discovery; Feature extraction; Image Processing and Computer Vision; Machine learning; Original Article; Probability and Statistics in Computer Science; Regression models; Aesthetics assessment; Transfer learning; Machine learning; Prediction; Classification; Feature extraction; Hybrid model; Convolutional neural networks
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-019-04065-4

The automatic assessment of the aesthetic value of an image is a task with many applications but really complex and challenging, due to the subjective component of the aesthetics for humans. The computational systems that carry out this task are usually composed of a set of ad hoc metrics proposed by the researchers and a machine learning system. We propose a new approach that fully automates the metrics creation process, its filtering and adjustment without human subjectivity. Thus, it does not depend on the authors’ human aesthetic intuitions. Our proposal is therefore based on the integration of two machine learning algorithms: CNN, which works as a feature extractor, and Correlation by Genetic Search (CGS)—a novel regression method, working as a supervised learning method. CGS is based on the creation of an adjusted linear regression model using Pearson’s correlation as a measure of performance in an evolutionary process. Experiments were conducted on a very well-known aesthetics database called “Photo.net” with more than a million images from over 400,000 users. The comparison of results with other approaches using the same dataset demonstrates that the fusion of CNN transfer learning features with this specific machine learning method has achieved robust and significantly better results than other state-of-the-art methods and hybrid approaches in terms of AUROC (0.93), accuracy (0.93) and Pearson’s correlation value (0.94).