Residual Recurrent Neural Network with Sparse Training for Offline Arabic Handwriting Recognition

Deep Recurrent Neural Networks (RNN) have been suffering from the overfitting problem due to the model redundancy of the network structures. We propose a novel temporal and spatial residual learning method for RNN, followed with sparse training by weight pruning to gain sparsity in network parameter...

Full description

Saved in:
Bibliographic Details
Published in2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) Vol. 1; pp. 1031 - 1037
Main Authors Ruijie Yan, Liangrui Peng, GuangXiang Bin, Shengjin Wang, Yao Cheng
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.11.2017
Subjects
Error analysis
Hidden Markov models
Mathematical model
Recurrent neural networks
Training
Online AccessGet full text

Cover

More Information
Summary:Deep Recurrent Neural Networks (RNN) have been suffering from the overfitting problem due to the model redundancy of the network structures. We propose a novel temporal and spatial residual learning method for RNN, followed with sparse training by weight pruning to gain sparsity in network parameters. For a Long Short-Term Memory (LSTM) network, we explore the combination schemes and parameter settings for temporal and spatial residual learning with sparse training. Experiments are carried out on the IFN/ENIT database. For the character error rate on the testing set e while training with sets a, b, c, d, the previously reported best result is 13.42%, and the proposed configuration of temporal residual learning followed with sparse training achieves the state-of-the-art result 12.06%.
ISSN:2379-2140
DOI:10.1109/ICDAR.2017.171