Combining weighted category-aware contextual information in convolutional neural networks for text classification

• Published in: World wide web (Bussum) Vol. 23; no. 5; pp. 2815 - 2834
• Main Authors: Wu, Xin, Cai, Yi, Li, Qing, Xu, Jingyun, Leung, Ho-fung
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2020; Springer Nature B.V
• Subjects: Artificial neural networks; Classification; Computer Science; Database Management; Information Systems Applications (incl.Internet); Natural language processing; Neural networks; Operating Systems; Special Issue on Web Information Systems Engineering 2018; Weighting methods; Convolutional neural networks; Word representation; Contextual information; Text classification
• ISSN: 1386-145X; 1573-1413
• DOI: 10.1007/s11280-019-00757-y

Convolutional neural networks (CNNs) are widely used in many natural language processing tasks, which employ some convolutional filters to capture useful semantic features of a text. However, a small window size convolutional filter is short of the ability to capture contextual information, simply increasing the window size may bring the problems of data sparsity and enormous parameters. To capture the contextual information, we propose to use the weighted sum operation to obtain contextual word representation. We present one implicit weighting method and two explicit category-aware weighting methods to assign the weights of the contextual information. Experimental results on five text classification datasets show the effectiveness of our proposed methods.