A multiscale dilated convolution and mixed-order attention-based deep neural network for monocular depth prediction

• Published in: SN applied sciences Vol. 5; no. 1; pp. 24 - 14
• Main Authors: Xu, Huihui, Li, Fei
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.01.2023; Springer Nature B.V; Springer
• Subjects: Algorithms; Applied and Technical Physics; Artificial neural networks; Attention mechanism; Cameras; Chemistry/Food Science; Classification; Computer applications; Convolution; Depth perception; Depth prediction; Design; Dilated convolution; Earth Sciences; Engineering; Environment; Feature enhancement; Feature extraction; Materials Science; Methods; Modules; Neural networks; Predictions; Receptive field; Research Article; Segmentation; Virtual reality; Feature enhancement; Attention mechanism; Dilated convolution; Depth prediction
• ISSN: 2523-3963; 2523-3971
• DOI: 10.1007/s42452-022-05235-1

Recovering precise depth information from different scenes has become a popular subject in the semantic segmentation and virtual reality fields. This study presents a multiscale dilated convolution and mixed-order attention-based deep neural network for monocular depth recovery. Specifically, we design a multilevel feature enhancement scheme to enhance and fuse high-resolution and low-resolution features on the basis of mixed-order attention. Moreover, a multiscale dilated convolution module that combines four different dilated convolutions is explored for deriving multiscale information and increasing the receptive field. Recent studies have shown that the design of loss terms is crucial to depth prediction. Therefore, an efficient loss function that combines the ℓ 1 loss, gradient loss, and classification loss is also designed to promote rich details. Experiments on three public datasets show that the presented approach achieves better performance than state-of-the-art depth prediction methods. Article Highlights We designed an efficient monocular depth prediction framework on the basis of multiscale dilated convolution and a mixed-order attention mechanism. This framework can produce effective depth outputs with rich details. Our mixed-order attention-based feature enhancement module can extract rich, meaningful contextual features in multiple stages, which further improves the feature representation ability. A dilated convolution-based multiscale strategy that can enlarge the receptive field is designed for obtaining meaningful multiscale features. We develop an efficient hybrid loss function including ℓ 1 loss, gradient loss, and classification loss to provide depth details.