MFE‐MVSNet: Multi‐scale feature enhancement multi‐view stereo with bi‐directional connections

• Published in: IET image processing Vol. 18; no. 11; pp. 2962 - 2973
• Main Authors: Lai, HongWei, Ye, ChunLong, Li, Zhenglin, Yan, Peng, Zhou, Yang
• Format: Journal Article
• Language: English
• Published: Wiley 01.09.2024
• Subjects: computer vision; image reconstruction; learning (artificial intelligence)
• ISSN: 1751-9659; 1751-9667
• DOI: 10.1049/ipr2.13147

Recent advancements in deep learning have significantly improved performance in the multi‐view stereo (MVS) domain, yet achieving a balance between reconstruction efficiency and quality remains challenging for learning‐based MVS methods. To address this, we introduce MFE‐MVSNet, designed for more effective and precise depth estimation. Our model incorporates a pyramid feature extraction network, featuring efficient multi‐scale attention and multi‐scale feature enhancement modules. These components capture pixel‐level pairwise relationships and semantic features with long‐range contextual information, enhancing feature representation. Additionally, we propose a lightweight 3D UNet regularization network based on depthwise separable convolutions to reduce computational costs. This network employs bi‐directional skip connections, establishing a fluid relationship between encoders and decoders and enabling cyclic reuse of building blocks without adding learnable parameters. By integrating these methods, MFE‐MVSNet effectively balances reconstruction quality and efficiency. Extensive qualitative and quantitative experiments on the DTU dataset validate our model's competitiveness, demonstrating approximately 33% and 12% relative improvements in overall score compared to MVSNet and CasMVSNet, respectively. Compared to other MVS networks, our approach more effectively balances reconstruction quality with efficiency. We introduce MFE‐MVSNet, which is designed for more effective and precise depth estimation. According to experiments, when compared to other multi‐view stereo networks, our approach more effectively balances reconstruction quality with efficiency.