GarmentGAN: Photo-realistic Adversarial Fashion Transfer

• Published in: 2020 25th International Conference on Pattern Recognition (ICPR) pp. 3923 - 3930
• Main Authors: Raffiee, Amir Hossein, Sollami, Michael
• Format: Conference Proceeding
• Language: English
• Published: IEEE 10.01.2021
• Subjects: Image segmentation; Image synthesis; Performance evaluation; Pipelines; Shape; Training
• DOI: 10.1109/ICPR48806.2021.9412908

The garment transfer problem comprises two tasks: learning to separate a person's body (pose, shape, color) from their clothing (garment type, shape, style) and then generating new images of the wearer dressed in arbitrary garments. We present GarmentGAN, a new algorithm that performs image-based garment transfer through generative adversarial methods. The GarmentGAN framework allows users to virtually try-on items before purchase and generalizes to various apparel types. GarmentGAN takes two images as input: a picture of the target fashion item and an image containing the customer. The output is a synthetic image wherein the customer is wearing the target apparel. To make the generated image look photo-realistic, we employ the use of novel generative adversarial techniques [18]. GarmentGAN improves on existing methods in the realism of generated imagery and solves various problems related to self-occlusions. Our proposed model incorporates additional information during training, utilizing both segmentation maps and body keypoint information. We demonstrate GarmentGAN outperforms current fashion transfer techniques through qualitative evaluation and standard image synthesis metrics.