Seamless Video Stitching from Hand-held Camera Inputs

• Published in: Computer graphics forum Vol. 35; no. 2; pp. 479 - 487
• Main Authors: Lin, Kaimo, Liu, Shuaicheng, Cheong, Loong-Fah, Zeng, Bing
• Format: Journal Article
• Language: English
• Published: Oxford Blackwell Publishing Ltd 01.05.2016
• Subjects: Analysis; Cameras; Categories and Subject Descriptors (according to ACM CCS); Cellular telephones; Devices; Digital cameras; Digital imaging; I.3.3 [Computer Graphics]: Picture/Image Generation-Viewing Algorithms; I.4.8 [Image Processing and Computer Vision]: Scene Analysis-Stereo; Image processing systems; Stitching; Studies; Three dimensional; Three dimensional imaging; Trajectories; Virtual cameras
• ISSN: 0167-7055; 1467-8659
• DOI: 10.1111/cgf.12848

Images/videos captured by portable devices (e.g., cellphones, DV cameras) often have limited fields of view. Image stitching, also referred to as mosaics or panorama, can produce a wide angle image by compositing several photographs together. Although various methods have been developed for image stitching in recent years, few works address the video stitching problem. In this paper, we present the first system to stitch videos captured by hand‐held cameras. We first recover the 3D camera paths and a sparse set of 3D scene points using CoSLAM system, and densely reconstruct the 3D scene in the overlapping regions. Then, we generate a smooth virtual camera path, which stays in the middle of the original paths. Finally, the stitched video is synthesized along the virtual path as if it was taken from this new trajectory. The warping required for the stitching is obtained by optimizing over both temporal stability and alignment quality, while leveraging on 3D information at our disposal. The experiments show that our method can produce high quality stitching results for various challenging scenarios.