High Fidelity Semantic Shape Completion for Point Clouds Using Latent Optimization

• Published in: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) pp. 1099 - 1108
• Main Authors: Gurumurthy, Swaminathan, Agrawal, Shubham
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.01.2019
• Subjects: Gallium nitride; Generative adversarial networks; Generators; Manifolds; Shape; Three-dimensional displays; Training
• DOI: 10.1109/WACV.2019.00122

Semantic shape completion is a challenging problem in 3D computer vision where the task is to generate a complete 3D shape using a partial 3D shape as input. We propose a learning-based approach to complete incomplete 3D shapes through generative modeling and latent manifold optimization. Our algorithm works directly on point clouds. We use an autoencoder and a GAN to learn a distribution of embeddings for point clouds of object classes. An input point cloud with missing regions is first encoded to a feature vector. The representations learnt by the GAN are then used to find the best latent vector on the manifold using a combined optimization that finds a vector in the manifold of plausible vectors that is close to the original input (both in the feature space and the output space of the decoder). Experiments show that our algorithm is capable of successfully reconstructing point clouds with large missing regions with very high fidelity without having to rely on exemplar based database retrieval.