LARGE-SCALE IMAGE SEARCH AND TAGGING USING IMAGE-TO-TOPIC EMBEDDING

• Main Authors: Zhang, Jianming, Jin, Hailin, Shen, Xiaohui, Li, Yingwei, Lin, Zhe
• Format: Patent
• Language: English
• Published: 11.11.2021
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

A framework is provided for associating images with topics utilizing embedding learning. The framework is trained utilizing images, each having multiple visual characteristics and multiple keyword tags associated therewith. Visual features are computed from the visual characteristics utilizing a convolutional neural network and an image feature vector is generated therefrom. The keyword tags are utilized to generate a weighted word vector (or "soft topic feature vector") for each image by calculating a weighted average of word vector representations that represent the keyword tags associated with the image. The image feature vector and the soft topic feature vector are aligned in a common embedding space and a relevancy score is computed for each of the keyword tags. Once trained, the framework can automatically tag images and a text based search engine can rank image relevance with respect to queried keywords based upon predicted relevancy scores. 10-USER 106- NETWORK 104 > If _--- IMAGE EMBEDDING SYSTEM 110d IMAGE TAG RECEIVING COMPONENT 108 IMAGE EMBEDDING VECTOR 112 GENERATING COMPONENT DATA SOFT TOPIC FEATURE VECTOR 114 GENERATING COMPONENT 116 ALIGNING COMPONENT