A hybrid method of combination probability and machine learning for Chinese geological text segmentation

• Published in: Computers & geosciences Vol. 183; p. 105512
• Main Authors: Guo, Zhiyong, Deng, Jiqiu, Zou, Yu, Tang, Yu
• Format: Journal Article
• Language: English
• Published: 01.01.2024
• Subjects: China; probability; surveys; China
• DOI: 10.1016/j.cageo.2023.105512
• ISSN: 0098-3004

To address the issues surrounding incomplete coverage of core dictionaries, limited training corpora, and low precision in Chinese geological text segmentation, a knowledge- and data-driven word segmentation method by combining combination probability and machine learning was proposed in this paper. We extracted mathematical feature information from terms in Chinese geological text to construct a Term Combination Probability Model (TCPM) for Chinese word combinations by integrating the combination features of geological terms and the Chinese writing styles under zero-sample conditions. The TCPM was used to extract geological terms with high combination characteristics as a user-defined dictionary, and then a geological corpus was constructed by using a general domain word segmentation method based on this dictionary. After a small amount of manual review and optimization, the geological corpus was trained with a BiLSTM-CRF model to segment Chinese geological text. The proposed method in this paper was tested using a regional geological survey report set in Henan Province, and the precision, recall, and F1-score of the method are 92.65%, 92.53%, and 92.59%, respectively. The experimental results demonstrated that this method, combined with the inherent knowledge features of geological text and the machine learning method, can assist in expanding the core dictionary for Chinese geological text segmentation based on zero-sample, and can improve the segmentation precision of Chinese geological text compared to simple general word segmentation methods or machine learning methods.