Uncalibrated, Two Source Photo-Polarimetric Stereo

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 44; no. 9; pp. 5747 - 5760
• Main Authors: Tozza, Silvia, Zhu, Dizhong, Smith, William A. P., Ramamoorthi, Ravi, Hancock, Edwin R.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.09.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Albedo; bas-relief ambiguity; Cameras; Combinatorial analysis; differential approach; Estimation; Illumination; illumination estimation; Image color analysis; Light sources; Lighting; Luminous intensity; Partial differential equations; photometric stereo; Photopolarimetry; Polarimetry; Polarisation; Polarization; Refractive index; Refractivity; Shape; shape-from-x
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2021.3078101

In this paper we present methods for estimating shape from polarisation and shading information, i.e. photo-polarimetric shape estimation, under varying, but unknown, illumination, i.e. in an uncalibrated scenario. We propose several alternative photo-polarimetric constraints that depend upon the partial derivatives of the surface and show how to express them in a unified system of partial differential equations of which previous work is a special case. By careful combination and manipulation of the constraints, we show how to eliminate non-linearities such that a discrete version of the problem can be solved using linear least squares. We derive a minimal, combinatorial approach for two source illumination estimation which we use with RANSAC for robust light direction and intensity estimation. We also introduce a new method for estimating a polarisation image from multichannel data and provide methods for estimating albedo and refractive index. We evaluate lighting, shape, albedo and refractive index estimation methods on both synthetic and real-world data showing improvements over existing state-of-the-art.