Mobile ID Document Recognition–Coarse-to-Fine Approach

• Published in: Pattern recognition and image analysis Vol. 32; no. 1; pp. 89 - 108
• Main Authors: Arlazarov, V. L., Arlazarov, V. V., Bulatov, K. B., Chernov, T. S., Nikolaev, D. P., Polevoy, D. V., Sheshkus, A. V., Skoryukina, N. S., Slavin, O. A., Usilin, S. A.
• Format: Journal Article
• Language: English
• Published: Moscow Pleiades Publishing 01.03.2022; Springer Nature B.V
• Subjects: Algorithms; Application Problems; Computer Science; Data processing; Documents; Electronic devices; Image classification; Image processing; Image Processing and Computer Vision; Image quality; Image segmentation; Noise reduction; Object recognition; Optical character recognition; Pattern Recognition; Template matching; Video data; optical character recognition; frame quality evaluation; document identification and matching; neural networks optimization; optical document recognition; real-time video processing; coarse-to-fine recognition; anytime algorithms
• ISSN: 1054-6618; 1555-6212
• DOI: 10.1134/S1054661822010023

Automatic optical recognition of documents is a traditional function of modern document processing systems. In this context, recognition represents a complex process which includes image processing, segmentation, classification, and linguistic analysis. Although the idea of using mobile devices for recognition of paper documents is not new, direct usage of existing software solutions for scanned images recognition yields low recognition precision on images obtained using a mobile device. This is due, first of all, to perspective distortions and lower effective resolution in the latter case. In this paper, we present an original approach and a set of algorithms for recognition of video frame sequence containing a document image, which is suitable for mobile implementation. It is based on a coarse-to-fine methodology, where template matching and fields localization are performed on the image with lowered resolution, followed by lazy processing of parts of the images only corresponding to the fields which are not recognized yet. Video stream is utilized as a source of noise reduction both in coordinates of the fields and optical character recognition classifiers outputs. The algorithm based on the proposed approach is suitable for running on the device itself and can operate even when none of the video frames contain a document image of sufficient quality by themselves.