Deep Learning on Image Stitching With Multi-viewpoint Images: A Survey

• Published in: Neural processing letters Vol. 55; no. 4; pp. 3863 - 3898
• Main Authors: Yan, Ni, Mei, Yupeng, Xu, Ling, Yu, Huihui, Sun, Boyang, Wang, Zimao, Chen, Yingyi
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.08.2023; Springer Nature B.V
• Subjects: Algorithms; Arrays; Artificial Intelligence; Cameras; Complex Systems; Computational Intelligence; Computer Science; Datasets; Deep learning; Distortion; Image acquisition; Neural networks; Review; Stitching; Deep learning; Deep homography estimation; Image matching; Multi-viewpoint; Image stitching; Image fusion
• ISSN: 1370-4621; 1573-773X
• DOI: 10.1007/s11063-023-11226-z

Multi-viewpoint image stitching aims to stitch images taken from different viewpoints into pictures with a broader field of view. The stitched images are subject to artifacts, geometric distortion, and blur distortion due to the mismatch of feature points, inaccurate homography estimation, and improper fusion of the unstitched images. Deep learning has recently been increasingly applied to multi-viewpoint image stitching to overcome these problems. However, there has thus far been little related research to summarize the different deep learning techniques used for multi-viewpoint image stitching. Therefore, this review aims to explore the application of deep learning to multi-viewpoint image stitching. To better illustrate this topic, we first summarize the acquisition methods for multi-viewpoint images and the main challenges of image stitching. After which, deep learning techniques for multi-view image stitching with a single camera are sorted out. Subsequently, deep learning techniques for multi-view image stitching with camera arrays, including parallel-view multi-view image stitching and cross-view multi-view image stitching, are presented. Next, we summarize image stitching datasets, evaluation metrics, and experimental data of several leading stitching algorithms on public datasets. Finally, we discuss potential issues and future work on image stitching with multi-viewpoint images.