IAMonSense: multi-level handwriting classification using spatiotemporal information

• Published in: International journal on document analysis and recognition Vol. 26; no. 3; pp. 303 - 319
• Main Authors: Mustafid, Ahmad, Younas, Junaid, Lukowicz, Paul, Ahmed, Sheraz
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2023; Springer Nature B.V
• Subjects: Annotations; Artificial neural networks; Benchmarks; Classification; Computer Science; Datasets; Enrichment; Handwriting; Handwriting recognition; Image Processing and Computer Vision; Neural networks; Pattern Recognition; Special Issue Paper; Deep learning; Transformers; Handwriting datasets; Graph neural networks; Spatiotemporal information; Handwriting classification
• ISSN: 1433-2833; 1433-2825
• DOI: 10.1007/s10032-023-00433-y

Online handwriting classification has become an open research problem as it serves as a preliminary step for handwriting recognition systems and applications in several other fields. This paper aims to extend the current trends and knowledge with multiple contributions in handwriting classification using spatiotemporal information. Firstly, it enriches the annotations of several publicly available online handwriting datasets, SenseThePen, IAM-onDB, and IAMonDo, for online handwriting classification and recognition tasks. The said datasets are updated with three distinguished levels of annotations, i.e., stroke, sequence, and line levels. The enriched annotations of these datasets extend their functionality for online handwriting classification at different levels for further research analysis. In addition to enrichment, it also unifies the annotation levels across the datasets, which enables the research community to benchmark proposed methods for comparative analysis using multiple datasets. All the datasets with enriched annotations are made publicly available for the research community as part of the IAMonSense dataset. Moreover, this paper presents a comprehensive benchmark of these datasets using multiple deep neural networks such as traditional convolutional neural networks (CNNs), graph convolutional networks(GCNs), attention-based neural networks, and transformers. These benchmarks can be used later on for further development in online handwriting classification.