Learning to Manipulate Individual Objects in an Image

We describe a method to train a generative model with latent factors that are (approximately) independent and localized. This means that perturbing the latent variables affects only local regions of the synthesized image, corresponding to objects. Unlike other unsupervised generative models, ours en...

Full description

Saved in:
Bibliographic Details
Published inProceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 6557 - 6566
Main Authors Yang, Yanchao, Chen, Yutong, Soatto, Stefano
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.01.2020
Subjects
Decoding
Image edge detection
Image reconstruction
Image segmentation
Mutual information
Semantics
Task analysis
Online AccessGet full text

Cover

More Information
Summary:We describe a method to train a generative model with latent factors that are (approximately) independent and localized. This means that perturbing the latent variables affects only local regions of the synthesized image, corresponding to objects. Unlike other unsupervised generative models, ours enables object-centric manipulation, without requiring object-level annotations, or any form of annotation for that matter. The key to our method is the combination of spatial disentanglement, enforced by a Contextual Information Separation loss, and perceptual cycle-consistency, enforced by a loss that penalizes changes in the image partition in response to perturbations of the latent factors. We test our method's ability to allow independent control of spatial and semantic factors of variability on existing datasets and also introduce two new ones that highlight the limitations of current methods.
ISSN:1063-6919
DOI:10.1109/CVPR42600.2020.00659