Theme-aware Visual Attribute Reasoning for Image Aesthetics Assessment

• Published in: IEEE transactions on circuits and systems for video technology Vol. 33; no. 9; p. 1
• Main Authors: Li, Leida, Huang, Yipo, Wu, Jinjian, Yang, Yuzhe, Li, Yaqian, Guo, Yandong, Shi, Guangming
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.09.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aesthetics; bilevel reasoning; Cognition; Data mining; Feature extraction; Image aesthetics assessment; Image color analysis; image theme; Lighting; Perception; Predictive models; Reasoning; visual attribute; Visualization
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2023.3249185

People usually assess image aesthetics according to visual attributes, e.g., interesting content, good lighting and vivid color, etc. Further, the perception of visual attributes depends on the image theme. Therefore, the inherent relationship between visual attributes and image theme is crucial for image aesthetics assessment (IAA), which has not been comprehensively investigated. With this motivation, this paper presents a new IAA model based on Theme-Aware Visual Attribute Reasoning (TAVAR). The underlying idea is to simulate the process of human perception in image aesthetics by performing bilevel reasoning. Specifically, a visual attribute analysis network and a theme understanding network are first pre-trained to extract aesthetic attribute features and theme features, respectively. Then, the first level Attribute-Theme Graph (ATG) is built to investigate the coupling relationship between visual attributes and image theme. Further, a flexible aesthetics network is introduced to extract general aesthetic features, based on which we built the second level Attribute-Aesthetics Graph (AAG) to mine the relationship between theme-aware visual attributes and aesthetic features, producing the final aesthetic prediction. Extensive experiments on four public IAA databases demonstrate the superiority of the proposed TAVAR model over the state-of-the-arts. Furthermore, TAVAR features better explainability due to the use of visual attributes.