Scene Uyghur Recognition Based on Visual Prediction Enhancement

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 20; p. 8610
• Main Authors: Liu, Yaqi, Kong, Fanjie, Xu, Miaomiao, Silamu, Wushour, Li, Yanbing
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 20.10.2023; MDPI
• Subjects: Analysis; correction network; Datasets; Deep learning; Language; Methods; scene text recognition; scene Uyghur dataset; Semantics; Uighurs; Uyghur recognition; vision model; China
• DOI: 10.3390/s23208610
• ISSN: 1424-8220

Aiming at the problems of Uyghur oblique deformation, character adhesion and character similarity in scene images, this paper proposes a scene Uyghur recognition model with enhanced visual prediction. First, the content-aware correction network TPS++ is used to perform feature-level correction for skewed text. Then, ABINet is used as the basic recognition network, and the U-Net structure in the vision model is improved to aggregate horizontal features, suppress multiple activation phenomena, better describe the spatial characteristics of character positions, and alleviate the problem of character adhesion. Finally, a visual masking semantic awareness (VMSA) module is added to guide the vision model to consider the language information in the visual space by masking the corresponding visual features on the attention map to obtain more accurate visual prediction. This module can not only alleviate the correction load of the language model, but also distinguish similar characters using the language information. The effectiveness of the improved method is verified by ablation experiments, and the model is compared with common scene text recognition methods and scene Uyghur recognition methods on the self-built scene Uyghur dataset.