Video Normals from Colored Lights

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 33; no. 10; pp. 2104 - 2114
• Main Authors: Brostow, G. J., Hernandez, C., Vogiatzis, G., Stenger, B., Cipolla, R.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.10.2011; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Artificial intelligence; Calibration; Cameras; Computer science; control theory; systems; Exact sciences and technology; Geometry; Image color analysis; Light sources; Lighting; multispectral; Pattern recognition. Digital image processing. Computational geometry; Photometric stereo; single view; Studies; Three dimensional displays; video normals; Computer vision; Multispectral detection; Tracking; Image processing; Video signal; Monocular vision; Silhouette; single view; Green function; Color image; Image reconstruction; multispectral; Structural unit; video normals; Image sequence; Photometric stereo; Tridimensional image; Stereopsis; Photometry
• ISSN: 0162-8828; 1939-3539; 2160-9292; 1939-3539
• DOI: 10.1109/TPAMI.2011.37

We present an algorithm and the associated single-view capture methodology to acquire the detailed 3D shape, bends, and wrinkles of deforming surfaces. Moving 3D data has been difficult to obtain by methods that rely on known surface features, structured light, or silhouettes. Multispectral photometric stereo is an attractive alternative because it can recover a dense normal field from an untextured surface. We show how to capture such data, which in turn allows us to demonstrate the strengths and limitations of our simple frame-to-frame registration over time. Experiments were performed on monocular video sequences of untextured cloth and faces with and without white makeup. Subjects were filmed under spatially separated red, green, and blue lights. Our first finding is that the color photometric stereo setup is able to produce smoothly varying per-frame reconstructions with high detail. Second, when these 3D reconstructions are augmented with 2D tracking results, one can register both the surfaces and relax the homogenous-color restriction of the single-hue subject. Quantitative and qualitative experiments explore both the practicality and limitations of this simple multispectral capture system.