What predicts citation counts and translational impact in headache research? A machine learning analysis

• Published in: Cephalalgia Vol. 44; no. 5; p. 3331024241251488
• Main Authors: Danelakis, Antonios, Langseth, Helge, Nachev, Parashkev, Nelson, Amy, Bjørk, Marte-Helene, Matharu, Manjit S., Tronvik, Erling, May, Arne, Stubberud, Anker
• Format: Journal Article
• Language: English
• Published: London, England SAGE Publications 01.05.2024
• Subjects: Area Under Curve; Authorship; Bibliometrics; Biomedical Research - statistics & numerical data; Datasets as Topic; Headache; Humans; Machine Learning; Periodicals as Topic; Practice Guidelines as Topic; Random Forest; ROC Curve; Translational Science, Biomedical - statistics & numerical data; prediction; deep learning; translational; Artificial intelligence; neural networks
• ISSN: 0333-1024; 1468-2982
• DOI: 10.1177/03331024241251488

Background We aimed to develop the first machine learning models to predict citation counts and the translational impact, defined as inclusion in guidelines or policy documents, of headache research, and assess which factors are most predictive. Methods Bibliometric data and the titles, abstracts, and keywords from 8600 publications in three headache-oriented journals from their inception to 31 December 2017 were used. A series of machine learning models were implemented to predict three classes of 5-year citation count intervals (0–5, 6–14 and, >14 citations); and the translational impact of a publication. Models were evaluated out-of-sample with area under the receiver operating characteristics curve (AUC). Results The top performing gradient boosting model predicted correct citation count class with an out-of-sample AUC of 0.81. Bibliometric data such as page count, number of references, first and last author citation counts and h-index were among the most important predictors. Prediction of translational impact worked optimally when including both bibliometric data and information from the title, abstract and keywords, reaching an out-of-sample AUC of 0.71 for the top performing random forest model. Conclusion Citation counts are best predicted by bibliometric data, while models incorporating both bibliometric data and publication content identifies the translational impact of headache research.