Restoring 2D Content from Distorted Documents

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 29; no. 11; pp. 1904 - 1916
• Main Authors: Brown, M.S., Mingxuan Sun, Ruigang Yang, Lin Yun, Seales, W.B.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.11.2007; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Automatic Data Processing - methods; Character recognition; Computer Graphics; Computer science; control theory; systems; Conformal mapping; Distortion; Document image processing; Document Processing; Document Restoration; Documentation - methods; Exact sciences and technology; Geometric Correction; Geometrical optics; Illumination; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image restoration; Information Storage and Retrieval - methods; Intelligence; Lighting; Nonlinear optics; Nonuniform; Numerical Analysis, Computer-Assisted; Optical character recognition; Optical character recognition software; Optical distortion; Optical variables control; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Photometric Correction; Reproducibility of Results; Sensitivity and Specificity; Shading Correction; Signal Processing, Computer-Assisted; Studies; Subtraction Technique; Three dimensional; Two dimensional; User-Computer Interface; Plane surface; Light source; High resolution; Image processing; Artefact; Conformal transformation; Document restoration; Document processing; geometric correction; Imaging; Illumination; Pattern analysis; Geometric transformation; Gradient; Image resolution; photometric correction; Text; Printed document; Character recognition; shading correction; Luminance; Optical character recognition; Document structure; conformal mapping; Tridimensional image; Artificial intelligence; Printed character; Reflectance
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2007.1118

This paper presents a framework to restore the 2D content printed on documents in the presence of geometric distortion and nonuniform illumination. Compared with text-based document imaging approaches that correct distortion to a level necessary to obtain sufficiently readable text or to facilitate optical character recognition (OCR), our work targets nontextual documents where the original printed content is desired. To achieve this goal, our framework acquires a 3D scan of the document's surface together with a high-resolution image. Conformal mapping is used to rectify geometric distortion by mapping the 3D surface back to a plane while minimizing angular distortion. This conformal "deskewing" assumes no parametric model of the document's surface and is suitable for arbitrary distortions. Illumination correction is performed by using the 3D shape to distinguish content gradient edges from illumination gradient edges in the high-resolution image. Integration is performed using only the content edges to obtain a reflectance image with significantly less illumination artifacts. This approach makes no assumptions about light sources and their positions. The results from the geometric and photometric correction are combined to produce the final output.