TCM-FTP: Fine-Tuning Large Language Models for Herbal Prescription Prediction

• Published in: arXiv.org
• Main Authors: Zhou, Xingzhi, Dong, Xin, Li, Chunhao, Bai, Yuning, Xu, Yulong, Cheung, Ka Chun, See, Simon, Song, Xinpeng, Zhang, Runshun, Zhou, Xuezhong, Zhang, Nevin L
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 15.07.2024
• Subjects: Chinese medicine; Computer Science - Artificial Intelligence; Computer Science - Computation and Language; Computer Science - Computational Engineering, Finance, and Science; Data augmentation; Datasets; Digestive system; Herbs; Large language models; Predictions; Prescriptions; Signs and symptoms; Traditional Chinese medicine
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.2407.10510

Traditional Chinese medicine (TCM) relies on specific combinations of herbs in prescriptions to treat symptoms and signs, a practice that spans thousands of years. Predicting TCM prescriptions presents a fascinating technical challenge with practical implications. However, this task faces limitations due to the scarcity of high-quality clinical datasets and the intricate relationship between symptoms and herbs. To address these issues, we introduce DigestDS, a new dataset containing practical medical records from experienced experts in digestive system diseases. We also propose a method, TCM-FTP (TCM Fine-Tuning Pre-trained), to leverage pre-trained large language models (LLMs) through supervised fine-tuning on DigestDS. Additionally, we enhance computational efficiency using a low-rank adaptation technique. TCM-FTP also incorporates data augmentation by permuting herbs within prescriptions, capitalizing on their order-agnostic properties. Impressively, TCM-FTP achieves an F1-score of 0.8031, surpassing previous methods significantly. Furthermore, it demonstrates remarkable accuracy in dosage prediction, achieving a normalized mean square error of 0.0604. In contrast, LLMs without fine-tuning perform poorly. Although LLMs have shown capabilities on a wide range of tasks, this work illustrates the importance of fine-tuning for TCM prescription prediction, and we have proposed an effective way to do that.