ClothFit: Cloth-Human-Attribute Guided Virtual Try-on Network Using 3D Simulated Dataset

• Published in: 2023 IEEE International Conference on Image Processing (ICIP) pp. 3484 - 3488
• Main Authors: Cho, Yunmin, Ray, Lala Shakti Swarup, Thota, Kundan Sai Prabhu, Suh, Sungho, Lukowicz, Paul
• Format: Conference Proceeding
• Language: English
• Published: IEEE 08.10.2023
• Subjects: Cloth simulation; Decoding; Digital apparel; Feature extraction; Garment fitting; Industries; Network architecture; Shape; Three-dimensional displays; Virtual try-on
• DOI: 10.1109/ICIP49359.2023.10222494

Online clothing shopping has become increasingly popular, but the high rate of returns due to size and fit issues has remained a major challenge. To address this problem, virtual try-on systems have been developed to provide customers with a more realistic and personalized way to try on clothing. In this paper, we propose a novel virtual try-on method called ClothFit, which can predict the draping shape of a garment on a target body based on the actual size of the garment and human attributes. Unlike existing try-on models, ClothFit considers the actual body proportions of the person and available cloth sizes for clothing virtualization, making it more appropriate for current online apparel outlets. The proposed method utilizes a U-Net-based network architecture that incorporates cloth and human attributes to guide the realistic virtual try-on synthesis. Specifically, we extract features from a cloth image using an auto-encoder and combine them with features from the user's height, weight, and cloth size. The features are concatenated with the features from the U-Net encoder, and the U-Net decoder synthesizes the final virtual try-on image. Our experimental results demonstrate that ClothFit can significantly improve the existing state-of-the-art methods in terms of photo-realistic virtual try-on results.