Physics-based learning: Adaptive structured light for active stereo depth estimation

• Published in: Optics and lasers in engineering Vol. 172; p. 107883
• Main Authors: Jia, Tong, Yang, Xiao, Liu, Yizhe, Li, Xiaofang, Chen, Dongyue, Deng, Shizhuo, Wang, Hao
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.01.2024
• Subjects: Adaptive structured light; Depth estimation; Physics-based learning; Depth estimation; Physics-based learning; Adaptive structured light
• ISSN: 0143-8166; 1873-0302
• DOI: 10.1016/j.optlaseng.2023.107883

Active stereo systems based on structured light are widely used for 3D vision in various applications, which project specially designed patterns onto object surfaces to encode each position in space for accurate 3D measurements. However, existing approaches use pre-determined patterns that are isolated from the scene properties (object reflectivity, distance, ambient light, inter-reflection), equipment (projector, camera), and reconstruction. Meanwhile, the parameters of the structured light are determined through empirical analysis or several experiments. In this paper, we propose a novel structured light design approach, named Physics-Based Learning Adaptive Structured Light (PBL-ASL), which directly learns the optimal structured light patterns from the scene. To this end, (1) we design a decoder with a non-sinusoidal error suppression module for PBL-ASL, which can accurately estimate the disparity during structured light optimization; (2) we propose a physics-based learning algorithm consisting of a self-supervised objective function and a differentiable imaging model, which computes the disparity error and back-propagates the gradient to the encoding vector to optimize the structured light. Our experiments demonstrate that PBL-ASL can significantly improve the depth estimation accuracy of active stereo systems over several state-of-art methods. •An adaptive structured light method for depth estimation is proposed.•Our method learns optimal structured light patterns directly from the scene.•We design a non-sinusoidal error suppression module.•We propose a self-supervised objective function and a differentiable imaging model.