Light Field Depth Estimation for Non-Lambertian Objects via Adaptive Cross Operator

• Published in: IEEE transactions on circuits and systems for video technology Vol. 34; no. 2; pp. 1199 - 1211
• Main Authors: Cui, Zhenglong, Sheng, Hao, Yang, Da, Wang, Sizhe, Chen, Rongshan, Ke, Wei
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.02.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: adaptive cross operator; Algorithms; Cameras; Costs; depth estimation; Depth measurement; Estimation; Image color analysis; Image reconstruction; Light field; Light fields; Multilayers; non-Lambertian; non-Lambertian region detection; Non-Lambertian surfaces; Separation; Three-dimensional displays
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2023.3292884

Light field (LF) depth estimation is a crucial basis for LF-related applications. Most existing methods are based on the Lambertian assumption and cannot deal with non-Lambertian surfaces represented by transparent objects and mirrors. In this paper, we propose a novel Adaptive-Cross-Operator-based(ACO) depth estimation algorithm for non-Lambertian LF. By analyzing the imaging characteristics of non-Lambertian regions, it is found that the difficulty of depth estimation lies in the photo inconsistency of the center view. Combining with the two-branch structure, we propose ACO with an inter-branch cooperation strategy to adaptively separate depth information with different reflectance coefficients. We discover that the bimodal distribution feature of the operator filtering results can assist in the separation of multi-layer scene information. The first detection branch filters the EPI and implicitly records the severity of multi-layer scene aliasing. According to the identification of bimodal distribution features, the non-Lambertian regions are marked out and the depth of the foreground is estimated. The second branch receives guidance from the first to dynamically adjust the inner weight and infer the background's depth after weakening the interference from the foreground. Finally, the depth information separation of multi-layer scenes is achieved by extracting the unique X-shaped linear structure. Without the reflection coefficients of the non-Lambertian object, the proposed method can produce high-quality depth estimation under the transparency of 90% to 20%. Experimental results show that the proposed ACO outperforms state-of-the-art LF depth estimation methods in terms of accuracy and robustness.