Fine-tuned Sentiment Analysis of COVID-19 Vaccine-Related Social Media Data: Comparative Study

• Published in: Journal of medical Internet research Vol. 24; no. 10; p. e40408
• Main Authors: Melton, Chad A, White, Brianna M, Davis, Robert L, Bednarczyk, Robert A, Shaban-Nejad, Arash
• Format: Journal Article
• Language: English
• Published: Canada Journal of Medical Internet Research 17.10.2022; Gunther Eysenbach MD MPH, Associate Professor; JMIR Publications
• Subjects: Analysis; Comparative analysis; Comparative studies; Computational linguistics; Coronaviruses; COVID-19; COVID-19 - prevention & control; COVID-19 Vaccines; Development and progression; Digital literacy; Educational materials; Educational systems; Epidemics; False information; Health behavior; Health education; Health information; Health promotion; Hugging; Humans; Immunization; Information sources; Internet; Language processing; Mass media; Medical research; Medicine, Experimental; Misinformation; Natural language interfaces; Original Paper; Pandemics; Psychological distress; Psychosocial factors; Public health; Public opinion; Sentiment Analysis; Social Media; Social networks; Social services delivery; Teams; Translation; Urgency; User statistics; Vaccination; Vaccines; information quality; natural language processing; misinformation; sentiment analysis; infodemiology; Twitter; COVID-19; vaccine; content analysis; Reddit; surveillance; DistilRoBERTa; social media; public health; vaccination
• ISSN: 1438-8871; 1439-4456; 1438-8871
• DOI: 10.2196/40408

The emergence of the novel coronavirus (COVID-19) and the necessary separation of populations have led to an unprecedented number of new social media users seeking information related to the pandemic. Currently, with an estimated 4.5 billion users worldwide, social media data offer an opportunity for near real-time analysis of large bodies of text related to disease outbreaks and vaccination. These analyses can be used by officials to develop appropriate public health messaging, digital interventions, educational materials, and policies. Our study investigated and compared public sentiment related to COVID-19 vaccines expressed on 2 popular social media platforms-Reddit and Twitter-harvested from January 1, 2020, to March 1, 2022. To accomplish this task, we created a fine-tuned DistilRoBERTa model to predict the sentiments of approximately 9.5 million tweets and 70 thousand Reddit comments. To fine-tune our model, our team manually labeled the sentiment of 3600 tweets and then augmented our data set through back-translation. Text sentiment for each social media platform was then classified with our fine-tuned model using Python programming language and the Hugging Face sentiment analysis pipeline. Our results determined that the average sentiment expressed on Twitter was more negative (5,215,830/9,518,270, 54.8%) than positive, and the sentiment expressed on Reddit was more positive (42,316/67,962, 62.3%) than negative. Although the average sentiment was found to vary between these social media platforms, both platforms displayed similar behavior related to the sentiment shared at key vaccine-related developments during the pandemic. Considering this similar trend in shared sentiment demonstrated across social media platforms, Twitter and Reddit continue to be valuable data sources that public health officials can use to strengthen vaccine confidence and combat misinformation. As the spread of misinformation poses a range of psychological and psychosocial risks (anxiety and fear, etc), there is an urgency in understanding the public perspective and attitude toward shared falsities. Comprehensive educational delivery systems tailored to a population's expressed sentiments that facilitate digital literacy, health information-seeking behavior, and precision health promotion could aid in clarifying such misinformation.