Efficient Agricultural Question Classification With a BERT-Enhanced DPCNN Model

• Published in: IEEE access Vol. 12; pp. 109255 - 109268
• Main Authors: Guo, Xiaojuan, Wang, Jianping, Gao, Guohong, Zhou, Junming, Li, Yancui, Cheng, Zihao, Miao, Guoyi
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Agricultural production; Agricultural short text; Artificial neural networks; Big Data; Classification; Computational modeling; contextual feature; Convolutional neural networks; Datasets; Feature extraction; fusion model; local feature; Performance enhancement; question classification; Questions; Semantics; Task analysis; Text categorization; Transformers; Words (language)
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3438848

The application of big data technology in agricultural production has led to explosive growth in agricultural data. The accurate classification of agricultural questions from vast amounts of question-and-answer data is currently a prominent topic in text classification research. However, due to the characteristics of agricultural questions, such as short text, high specialization, and uneven sample distribution, relying on a single model for feature extraction and classification has limitations. To address this issue and improve the performance of agricultural question classification, we propose the fusion text classification model BERT-DPCNN, which combines the Bidirectional Encoder Representations from Transformer (BERT) model with the Deep Pyramid Convolution Neural Network (DPCNN). Firstly, the BERT pre-training model captures word-level semantic information for each question and generates hidden vectors containing sentence-level features using 12 layers of transformers. Secondly, the output word vectors are input into DPCNN to further extract local features of the word-level text and capture long-distance textual dependencies. Finally, we verified the effectiveness of our fusion model using a self-constructed agricultural question dataset. Comparative experiments demonstrate that BERT-DPCNN achieves superior classification results with an accuracy rate of 99.07%. To assess its generalization performance, we conducted comparison experiments on the Tsinghua News dataset. Experimental results show significant improvement in BERT-DPCNN's classification performance on agricultural question datasets compared to other models, meeting requirements for question classification in agricultural question-answering systems.