Multilingual scene character recognition with co-occurrence of histogram of oriented gradients

• Published in: Pattern recognition Vol. 51; pp. 125 - 134
• Main Authors: Tian, Shangxuan, Bhattacharya, Ujjwal, Lu, Shijian, Su, Bolan, Wang, Qingqing, Wei, Xiaohua, Lu, Yue, Tan, Chew Lim
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.03.2016
• Subjects: Co-occurrence HOG; Histogram of Oriented Gradients; Multilingual Scene Text; Scene Character Recognition; Scene Character Recognition; Histogram of Oriented Gradients; Multilingual Scene Text; Co-occurrence HOG
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2015.07.009

Automatic machine reading of texts in scenes is largely restricted by the poor character recognition accuracy. In this paper, we extend the Histogram of Oriented Gradient (HOG) and propose two new feature descriptors: Co-occurrence HOG (Co-HOG) and Convolutional Co-HOG (ConvCo-HOG) for accurate recognition of scene texts of different languages. Compared with HOG which counts orientation frequency of each single pixel, the Co-HOG encodes more spatial contextual information by capturing the co-occurrence of orientation pairs of neighboring pixels. Additionally, ConvCo-HOG exhaustively extracts Co-HOG features from every possible image patches within a character image for more spatial information. The two features have been evaluated extensively on five scene character datasets of three different languages including three sets in English, one set in Chinese and one set in Bengali. Experiments show that the proposed techniques provide superior scene character recognition accuracy and are capable of recognizing scene texts of different scripts and languages. •Introduced powerful features Co-HOG and ConvCo-HOG for scene character recognition.•Designed a new offset based strategy for dimension reduction of above features.•Developed two new scene character datasets for Chinese and Bengali scripts.•Extensive simulations on 5 datasets of 3 scripts show the efficiency of the approach.