A Comparative Study of Short Text Classification with Spiking Neural Networks

• Published in: 2022 17th Conference on Computer Science and Intelligence Systems (FedCSIS) Vol. 30; pp. 79 - 88
• Main Authors: Maciag, Piotr S., Sitek, Wojciech, Skonieczny, Lukasz, Rybinski, Henryk
• Format: Conference Proceeding Journal Article
• Language: English
• Published: Polish Information Processing Society 01.01.2022
• Subjects: Computer science; Encoding; evolving spiking neural networks; medical documents; NeuCube; Neural networks; PubMed documents; short text classification; SNNTorch; spiking neural networks; Task analysis; Text categorization
• ISSN: 2300-5963
• DOI: 10.15439/2022F184

Short text classification is an important task widely used in many applications. However, few works investigated applying Spiking Neural Networks (SNNs) for text classification. To the best of our knowledge, there were no attempts to apply SNNs as classifiers of short texts. In this paper, we offer a comparative study of short text classification using SNNs. To this end, we selected and evaluated three popular implementations of SNNs: evolving Spiking Neural Networks (eSNN), the NeuCube implementation of SNNs, as well as the SNNTorch implementation that is available as the Python language package. In order to test the selected classifiers, we selected and preprocessed three publicly available datasets: 20-newsgroup dataset as well as imbalanced and balanced PubMed datasets of medical publications. The preprocessed 20-newsgroup dataset consists of first 100 words of each text, while for the classification of PubMed datasets we use only a title of each publication. As a text representation of documents, we applied the TF-IDF encoding. In this work, we also offered a new encoding method for eSNN networks, that can effectively encode values of input features having non-uniform distributions. The designed method works especially effectively with the TF-IDF encoding. The results of our study suggest that SNN networks may provide the classification quality is some cases matching or outperforming other types of classifiers.