As-Projective-As-Possible Image Stitching with Moving DLT

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 36; no. 7; pp. 1285 - 1298
• Main Authors: Zaragoza, Julio, Tat-Jun Chin, Tran, Quoc-Huy, Brown, Michael S., Suter, David
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.07.2014; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Artificial intelligence; Cameras; Computer science; control theory; systems; Direct Linear Transformation; Educational institutions; Estimation; Exact sciences and technology; Extrapolation; Image Alignment; Image Stitching; Moving Least Squares; Pattern recognition. Digital image processing. Computational geometry; Projective Warps; Three-dimensional displays; Yarn; image alignment; direct linear transformation; projective warps; Image stitching; moving least squares; Bundle adjustment; Image processing; Equity; Artefact; Image reconstruction; Multiple image; Alignment; Perspective; Image matching; Moving image; Least squares method; Image sequence; Imaging; Homograph; Computer vision; Composition; Stereoscopy; Realistic images; Linear transformation; Tridimensional image; Ghosting artefact; Alignment defect; Spatial representation
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2013.247

The success of commercial image stitching tools often leads to the impression that image stitching is a "solved problem". The reality, however, is that many tools give unconvincing results when the input photos violate fairly restrictive imaging assumptions; the main two being that the photos correspond to views that differ purely by rotation, or that the imaged scene is effectively planar. Such assumptions underpin the usage of 2D projective transforms or homographies to align photos. In the hands of the casual user, such conditions are often violated, yielding misalignment artifacts or "ghosting" in the results. Accordingly, many existing image stitching tools depend critically on post-processing routines to conceal ghosting. In this paper, we propose a novel estimation technique called Moving Direct Linear Transformation (Moving DLT) that is able to tweak or fine-tune the projective warp to accommodate the deviations of the input data from the idealized conditions. This produces as-projective-as-possible image alignment that significantly reduces ghosting without compromising the geometric realism of perspective image stitching. Our technique thus lessens the dependency on potentially expensive postprocessing algorithms. In addition, we describe how multiple as-projective-as-possible warps can be simultaneously refined via bundle adjustment to accurately align multiple images for large panorama creation.