Enhanced predicting genu valgum through integrated feature extraction: Utilizing ChatGPT with body landmarks

• Published in: Biomedical signal processing and control Vol. 97; p. 106676
• Main Authors: Meng, Deyu, He, Shichun, Wei, Meiqi, Lv, Zongnan, Guo, Hongzhi, Yang, Guang, Wang, Ziheng
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.11.2024
• Subjects: Body landmarks; Body posture; ChatGPT; Genu valgum; Non-contact detection; Body posture; Non-contact detection; Body landmarks; Genu valgum; ChatGPT
• ISSN: 1746-8094
• DOI: 10.1016/j.bspc.2024.106676

Genu valgum is a postural anomaly that affects the alignment and functionality of the lower extremity. However, current methods for measuring genu valgum have limitations such as inconsistency, invasiveness, cost and inconvenience. Therefore, this study aims to develop a deep learning architecture for the prediction of genu valgum using non-contact pose analysis. After collating posture data from 1519 Chinese adolescents, we developed a model with two branches: one branch uses Real-Time Multi-Person Pose (RTM Pose) to identify landmarks related to genu valgum, and another branch involves ChatGPT, which generates supplementary features by receiving prompts and inputting subject images. In addition, to verify whether ChatGPT provided useful features, we compared it against a baseline model that used only RTM Pose. Our model aspires to provide a comprehensive perspective, potentially elevating the accuracy in detecting and prognosticating genu valgum. Fortunately, our model outperformed the baseline, achieving a 77.19% accuracy. This demonstrates that ChatGPT can effectively capture semantic information from images, providing a novel and useful feature for the detection of genu valgum. Our study presents a rapid and effective means of predicting genu valgum using non-contact pose analysis and highlights the potential of ChatGPT in medical imaging. •Keypoint and ChatGPT-based features accurately identify genu valgum in adolescents.•Non-contact and non-X-ray detection method achieves the better performance.•The method we proposed achieves an accuracy of 77.19%, recall of 77.00% and auc of 83.04%.