Single-View 3D Shape Reconstruction with Learned Gradient Descent

• Published in: Pattern Recognition and Computer Vision pp. 173 - 184
• Main Authors: Zhang, Guanglun, Yang, Lu
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: 3D shape reconstruction; Deep learning; Learned gradient descent
• ISBN: 303060635X; 9783030606350
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-60636-7_15

Reconstructing the 3D shape from single image has become a popular research topic imputed to the end-to-end learning ability of deep convolutional networks. In this paper, we show that, the 3D-2D geometry knowledge can be explicitly incorporated into the deep convolutional network to regularize the reconstruction task. Leveraging recent advances in learned gradient descent, we pass the gradient components directly to the learning network during learning to enable a sequence of update CNNs, which can generate updates to the predicted 3D shape. Hence, we can explicitly regularize the learnable 3D reconstruction with the projective constraint between 2D view and 3D shape. We show that our method can outperform the state-of-the-art results on the ShapeNet test dataset as our network has learned a 2D-3D prior.