Making one object look like another: controlling appearance using a projector-camera system

• Published in: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004 Vol. 1; p. I
• Main Authors: Grossberg, M.D., Peri, H., Nayar, S.K., Belhumeur, P.N.
• Format: Conference Proceeding
• Language: English
• Published: Los Alamitos, California IEEE 2004; IEEE Computer Society
• Subjects: Applied sciences; Artificial intelligence; Calibration; Cameras; Color; Computer displays; Computer science; Computer science; control theory; systems; Control systems; Exact sciences and technology; Lighting; Mechanical engineering. Machine design; Pattern recognition. Digital image processing. Computational geometry; Product design; Radiometry; Reflectivity; Spectral response; Computer vision; Entertainment; Image processing; Calibration; Pattern recognition; Spatial variation; Adaptive method; Teaching; Interactive system; Engineering design; Direct method; Image projection; Camouflage; Education; User interface; Lighting; Distraction; Environmental effect; Product development; Reflectance
• ISBN: 9780769521589; 0769521584
• ISSN: 1063-6919
• DOI: 10.1109/CVPR.2004.1315067

We present a method for controlling the appearance of an arbitrary 3D object using a projector and a camera. Our goal is to make one object look like another by projecting a carefully determined compensation image onto the object. The determination of the appropriate compensation image requires accounting for spatial variation in the object's reflectance, the effects of environmental lighting, and the spectral responses, spatially varying fall-offs, and non-linear responses in the projector-camera system. Addressing each of these effects, we present a compensation method which calls for the estimation of only a small number of parameters, as part of a novel off-line radiometric calibration. This calibration is accomplished by projecting and acquiring a minimal set of 6 images, irrespective of the object. Results of the calibration are then used on-line to compensate each input image prior to projection. Several experimental results are shown that demonstrate the ability of this method to control the appearance of everyday objects. Our method has direct applications in several areas including smart environments, product design and presentation, adaptive camouflages, interactive education and entertainment.