Fine-Grained Visual Attribute Extraction from Fashion Wear

• Published in: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) pp. 3968 - 3972
• Main Authors: Parekh, Viral, Shaik, Karimulla, Biswas, Soma, Chelliah, Muthusamy
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2021
• Subjects: Computer architecture; Computer vision; Conferences; Data mining; Footwear; Neck; Visualization
• ISSN: 2160-7516
• DOI: 10.1109/CVPRW53098.2021.00447

Automatically extracting visual attributes for ecommerce data has widespread applications in cataloging, catalogue qualification and enrichment, visual search, etc. Here, we address the task of visual attribute extraction for a highly challenging real-world fashion data from Flipkart catalogue (an Indian e-commerce platform), which is collected from seller uploaded product images. This data not only contains widely varying categories (e.g., shirt, sari, shoes), but also has both coarse-grained (e.g., occasion, top type, sari type) and fine-grained (e.g., neck type, print type) attributes. Training examples available for different attributes are highly imbalanced, making this task even more challenging. To this end, we propose an end-to-end framework which integrates multi-task learning with transformer as an attention module, in addition to handling the data imbalance. The proposed architecture supports multiple attributes across various product categories in a scalable manner. Extensive experiments on the in-house dataset shows effectiveness of the proposed framework in improving performance of the fine-grained attributes by 13% on the baseline across the attributes.