Optimization on machine learning based approaches for sentiment analysis on HPV vaccines related tweets

• Published in: Journal of biomedical semantics Vol. 8; no. 1; p. 9
• Main Authors: Du, Jingcheng, Xu, Jun, Song, Hsingyi, Liu, Xiangyu, Tao, Cui
• Format: Journal Article
• Language: English
• Published: London BioMed Central 03.03.2017; BioMed Central Ltd
• Subjects: Algorithms; Attitude to Health; Bioinformatics; Combinatorial Libraries; Computational Biology/Bioinformatics; Computer Appl. in Life Sciences; Data Mining; Data Mining and Knowledge Discovery; Emotions; Humans; Machine Learning; Mathematics; Mathematics and Statistics; Papillomavirus Vaccines; Public opinion; Social Media; Vaccination; Vaccine and drug ontology in the study of mechanism and effect (VDOSME); Support vector machines; Sentiment analysis; Twitter; Gold standard; Hierarchical classification; Social media
• ISSN: 2041-1480; 2041-1480
• DOI: 10.1186/s13326-017-0120-6

Background Analysing public opinions on HPV vaccines on social media using machine learning based approaches will help us understand the reasons behind the low vaccine coverage and come up with corresponding strategies to improve vaccine uptake. Objective To propose a machine learning system that is able to extract comprehensive public sentiment on HPV vaccines on Twitter with satisfying performance. Method We collected and manually annotated 6,000 HPV vaccines related tweets as a gold standard. SVM model was chosen and a hierarchical classification method was proposed and evaluated. Additional feature sets evaluation and model parameters optimization was done to maximize the machine learning model performance. Results A hierarchical classification scheme that contains 10 categories was built to access public opinions toward HPV vaccines comprehensively. A 6,000 annotated tweets gold corpus with Kappa annotation agreement at 0.851 was created and made public available. The hierarchical classification model with optimized feature sets and model parameters has increased the micro-averaging and macro-averaging F score from 0.6732 and 0.3967 to 0.7442 and 0.5883 respectively, compared with baseline model. Conclusions Our work provides a systematical way to improve the machine learning model performance on the highly unbalanced HPV vaccines related tweets corpus. Our system can be further applied on a large tweets corpus to extract large-scale public opinion towards HPV vaccines.