Automatic tag expansion using visual similarity for photo sharing websites

• Published in: Multimedia tools and applications Vol. 49; no. 1; pp. 81 - 99
• Main Authors: Sevil, Sare Gul, Kucuktunc, Onur, Duygulu, Pinar, Can, Fazli
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.08.2010; Springer Nature B.V
• Subjects: Computer Communication Networks; Computer Science; Cues; Data Structures and Information Theory; Image retrieval; Information management; Joining; Lists; Metadata; Methods; Multimedia; Multimedia Information Systems; Similarity; Social networks; Special Purpose and Application-Based Systems; Studies; Tagging; Tags; Visual; Websites; Tagging; Photo-annotation; Flickr; Visual similarity; Folksonomy
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-009-0394-5

In this paper we present an automatic photo tag expansion method designed for photo sharing websites. The purpose of the method is to suggest tags that are relevant to the visual content of a given photo at upload time. Both textual and visual cues are used in the process of tag expansion. When a photo is to be uploaded, the system asks for a couple of initial tags from the user. The initial tags are used to retrieve relevant photos together with their tags. These photos are assumed to be potentially content related to the uploaded target photo. The tag sets of the relevant photos are used to form the candidate tag list, and visual similarities between the target photo and relevant photos are used to give weights to these candidate tags. Tags with the highest weights are suggested to the user. The method is applied on Flickr ( http://www.flickr.com ). Results show that including visual information in the process of photo tagging increases accuracy with respect to text-based methods.