Multimodal Web Aesthetics Assessment Based on Structural SVM and Multitask Fusion Learning

• Published in: IEEE transactions on multimedia Vol. 18; no. 6; pp. 1062 - 1076
• Main Authors: Wu, Ou, Zuo, Haiqiang, Hu, Weiming, Li, Bing
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.06.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aesthetics; Feature extraction; Human computer interaction; Image color analysis; local features; Measurement; Usability; Visual aesthetics; Visualization; Web pages; fusion; local features; visual aesthetics; multi-task learning; Aesthetic features; web pages
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2016.2538722

The overall visual attributes (e.g., aesthetics) of Web pages significantly influence user experience. A beautiful and well laid out Web page greatly facilitates user access and enhances the browsing experience. In this paper, a new method is proposed to learn an assessment model for the (visual) aesthetics of Web pages. First, multimodal features (structural, local visual, global visual, and functional) of a Web page that are known to significantly affect the aesthetics of a Web page are extracted to construct a feature vector. Second, the interuser disagreement of aesthetics is analyzed and novel aesthetic representations are obtained from the multiuser ratings of a page. A structural learning algorithm is proposed for the new aesthetic representations. Third, as a Web page's functional purpose also affects the perceived aesthetics, we divide Web pages into different types using functional features, and a soft multitask fusion learning strategy is introduced to train assessment models for pages with functional purposes. Experimental results show the effectiveness of our method: 1) the combination of structural, local, and global visual features outperforms existing state-of-the-art Web aesthetic features; 2) the proposed structural learning algorithm achieves good results for the new aesthetic representations; and 3) the proposed soft multitask fusion learning strategy improves the performances of aesthetics assessment models.