Progressive generation of 3D point clouds with hierarchical consistency

• Published in: Pattern recognition Vol. 136; p. 109200
• Main Authors: Li, Peipei, Liu, Xiyan, Huang, Jizhou, Xia, Deguo, Yang, Jianzhong, Lu, Zhen
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2023
• Subjects: 3D Point cloud generation; Generative adversarial networks; Hierarchical consistency; Point cloud analysis; Variational autoencoder; Variational autoencoder; Generative adversarial networks; Point cloud analysis; 3D Point cloud generation; Hierarchical consistency
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2022.109200

•We propose an effective generative model named hierarchical consistency variational autoencoder (HC-VAE) for the point cloud generation. Our model can generate dense point clouds of diverse shapes with uniformly distributed points in an unsupervised manner. We also hope that this research can be a stepping stone to more feasible hierarchical distributions-based generalization.•Hierarchical consistent mechanism (HCM) is presented to crisply constrain the model from both the shape distribution and the pointwise distribution, respectively. The hierarchical constraints seamlessly combine the learning of shape distribution and pointwise distribution in a complementary fashion, thereby facilitating the models perception of 3D shape and desensitization of the distribution of points given a shape.•Extensive qualitative and quantitative experiments demonstrate that HC-VAE achieves state-of-the-art performance in the 3D point cloud generation community with pleasing local topology details and distinguishable global shape information Generating 3D point cloud directly from latent prior (e.g., Gaussian distribution) plays a vital role in the representation learning and data augmentation in 3D vision tasks. Since point cloud is formed by irregular points, the generation process of point cloud requires rich semantic information, yet few studies are devoted to it. In this paper, we recast this generation task as a progressive learning problem to model the two-level hierarchy of distributions and address it by proposing a novel model called hierarchical consistency variational autoencoder (HC-VAE). This framework introduces a hierarchical consistent mechanism (HCM) to model the shape consistency and the pointwise representation consistency in a complementary manner. Specifically, we propose a stackable encoder-decoder framework and constrain the generation quality progressively to ensure that the underlying shape and fine-grained parts can be reconstructed with high fidelity. Additionally, given the progressively generated intermediate point cloud instances, a hierarchical-positive contrastive loss is introduced to learn the point-distribution-free instance representations to avoid explicitly parametrizing the distribution of points in a shape. In this way, our model suffices to generate diverse, high-resolution, and uniform point cloud instances. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance in point cloud generation.