Deep saliency models : The quest for the loss function

• Published in: Neurocomputing (Amsterdam) Vol. 453; pp. 693 - 704
• Main Authors: Bruckert, Alexandre, Tavakoli, Hamed R., Liu, Zhi, Christie, Marc, Le Meur, Olivier
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 17.09.2021; Elsevier
• Subjects: Computer Science; Computer Vision and Pattern Recognition; Deep saliency models; Human visual saliency; Image Processing; Loss functions; Neural and Evolutionary Computing; Loss functions; Human visual saliency; Deep saliency models
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2020.06.131

Deep learning techniques are widely used to model human visual saliency, to such a point that state-of-the-art performances are now only attained by deep neural networks. However, one key part of a typical deep learning model is often neglected when it comes to modeling visual saliency: the choice of the loss function. In this work, we explore some of the most popular loss functions that are used in deep saliency models. We demonstrate that on a fixed network architecture, modifying the loss function can significantly improve (or depreciate) the results, hence emphasizing the importance of the choice of the loss function when designing a model. We also evaluate the relevance of new loss functions for saliency prediction inspired by metrics used in style-transfer tasks. Finally, we show that a linear combination of several well-chosen loss functions leads to significant improvements in performance on different datasets as well as on a different network architecture, thus demonstrating the robustness of a combined metric.