Fast shape-from-template using local features

• Published in: Machine vision and applications Vol. 29; no. 1; pp. 73 - 93
• Main Authors: Famouri, Mahmoud, Bartoli, Adrien, Azimifar, Zohreh
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.01.2018; Springer Nature B.V; Springer Verlag
• Subjects: Affine transformations; Bioengineering; Communications Engineering; Computer Science; Computer vision; Data analysis; Deformation; Formability; Image Processing and Computer Vision; Image reconstruction; Life Sciences; Networks; Original Paper; Outliers (statistics); Pattern Recognition; Three dimensional models; Vision systems; Fast shape-from-template; Affine transformation estimation; Depth estimation; Local texture; Deformable surface; Geometrical outlier removal
• ISSN: 0932-8092; 1432-1769
• DOI: 10.1007/s00138-017-0876-9

Reconstructing the 3D surface of an object using only a single image is a challenging task, which has recently attracted attention. In this paper, a template-based approach is presented to reconstruct the surface of an isometric deformable object. The proposed approach brings a solution for a class of computer vision problems named shape-from-template (SfT). In SfT, the goal is to solve single-image reconstruction for an object given its 3D template model in some rest shape. To this end, corresponding keypoints between the template and the so-called deformed image are first established. Then, a very fast method is used to estimate the first-order differential flow around the extracted keypoint pairs as an affine transformation. This is done using the keypoint pairs’ surrounding texture patch. In our method, we estimate this affine transformation using the keypoint pairs’ closest neighbors. This is both faster and more stable. Finally, the depth of each keypoint in the deformed image is estimated from its associated affine transformation. The robustness of keypoint matching is essential to the process. Indeed, outliers defeat depth estimation dramatically. We propose two new approaches to detect and remove the possible outliers based on geometrical properties of the matched keypoints. These two geometrical outlier removal approaches are faster than existing ones and can be used with almost any image descriptor. Experimental results show that the proposed approaches are very effective and outperform existing ones.