A Plug-in Method for Representation Factorization in Connectionist Models

• Published in: IEEE transaction on neural networks and learning systems Vol. 33; no. 8; pp. 3792 - 3803
• Main Authors: Yoon, Jee Seok, Roh, Myung-Cheol, Suk, Heung-Il
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Ablation; Computer vision; Correlation; Decomposition; Factorization; few-shot learning; Gallium nitride; Generative adversarial networks; image-to-image translation; Information theory; Learning systems; Mutual information; Neural networks; Plugs; representation learning; Representations; Semantics; style transfer; Task analysis
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3054480

In this article, we focus on decomposing latent representations in generative adversarial networks or learned feature representations in deep autoencoders into semantically controllable factors in a semisupervised manner, without modifying the original trained models. Particularly, we propose factors' decomposer-entangler network (FDEN) that learns to decompose a latent representation into mutually independent factors. Given a latent representation, the proposed framework draws a set of interpretable factors, each aligned to independent factors of variations by minimizing their total correlation in an information-theoretic means. As a plug-in method, we have applied our proposed FDEN to the existing networks of adversarially learned inference and pioneer network and performed computer vision tasks of image-to-image translation in semantic ways, e.g., changing styles, while keeping the identity of a subject, and object classification in a few-shot learning scheme. We have also validated the effectiveness of the proposed method with various ablation studies in the qualitative, quantitative, and statistical examination.