Multi-Label Classification of Historical Documents by Using Hierarchical Attention Networks

• Published in: Journal of the Korean Physical Society Vol. 76; no. 5; pp. 368 - 377
• Main Authors: Kim, Dong-Kyum, Lee, Byunghwee, Kim, Daniel, Jeong, Hawoong
• Format: Journal Article
• Language: English
• Published: Seoul The Korean Physical Society 01.03.2020; Springer Nature B.V; 한국물리학회
• Subjects: Algorithms; Categories; Classification; Historians; Keywords; Machine learning; Mathematical and Computational Physics; Particle and Nuclear Physics; Physics; Physics and Astronomy; Quantitative analysis; Theoretical; 물리학; Deep learning; Text analysis; Big data; Recurrent neural network; History
• ISSN: 0374-4884; 1976-8524
• DOI: 10.3938/jkps.76.368

The quantitative analysis of digitized historical documents has begun in earnest in recent years. Text classification is of particular importance for quantitative historical analysis because it helps to search literature efficiently and to determine the important subjects of a particular age. While numerous historians have joined together to classify large-scale historical documents, consistent classification among individual researchers has not been achieved. In this study, we present a classification method for large-scale historical data that uses a recently developed supervised learning algorithm called the Hierarchical Attention Network (HAN). By applying various classification methods to the Annals of the Joseon Dynasty (AJD), we show that HAN is more accurate than conventional techniques with word-frequency-based features. HAN provides the extent that a particular sentence or word contributes to the classification process through a quantitative value called ’attention’. We extract the representative keywords from various categories by using the attention mechanism and show the evolution of the keywords over the 472-year span of the AJD. Our results reveal that largely two groups of event categories are found in the AJD. In one group, the representative keywords of the categories were stable over long periods while the keywords in the other group varied rapidly, exhibiting repeatedly changing characteristics of the categories. Observing such macroscopic changes of representative words may provide insight into how a particular topic changes over a historical period.