Energy-Efficient Adaptive 3D Sensing

• Published in: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 5054 - 5063
• Main Authors: Tilmon, Brevin, Sun, Zhanghao, Koppal, Sanjeev J., Wu, Yicheng, Evangelidis, Georgios, Zahreddine, Ramzi, Krishnan, Gurunandan, Ma, Sizhuo, Wang, Jian
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2023
• Subjects: Computational imaging; Estimation; Optical diffraction; Phase modulation; Power demand; Prototypes; Robot sensing systems; Three-dimensional displays
• ISSN: 2575-7075
• DOI: 10.1109/CVPR52729.2023.00489

Active depth sensing achieves robust depth estimation but is usually limited by the sensing range. Naively increasing the optical power can improve sensing range but induces eye-safety concerns for many applications, including autonomous robots and augmented reality. In this paper, we propose an adaptive active depth sensor that jointly optimizes range, power consumption, and eye-safety. The main observation is that we need not project light patterns to the entire scene but only to small regions of interest where depth is necessary for the application and passive stereo depth estimation fails. We theoretically compare this adaptive sensing scheme with other sensing strategies, such as full-frame projection, line scanning, and point scanning. We show that, to achieve the same maximum sensing distance, the proposed method consumes the least power while having the shortest (best) eye-safety distance. We implement this adaptive sensing scheme with two hardware prototypes, one with a phase-only spatial light modulator (SLM) and the other with a micro-electro-mechanical (MEMS) mirror and diffractive optical elements (DOE). Experimental results validate the advantage of our method and demonstrate its capability of acquiring higher quality geometry adaptively. Please see our project website for video results and code: https://btilmon.github.io/e3d.html.