Topic modelling for qualitative studies

• Published in: Journal of information science Vol. 43; no. 1; pp. 88 - 102
• Main Authors: Nikolenko, Sergey I., Koltcov, Sergei, Koltsova, Olessia
• Format: Journal Article
• Language: English
• Published: London, England SAGE Publications 01.02.2017; Bowker-Saur Ltd
• Subjects: Coherence; Data mining; Datasets; Dirichlet problem; Judgment; Mathematical models; Modelling; Qualitative analysis; Qualitative research; Latent Dirichlet allocation; LDA extensions; topic modelling; topic quality
• ISSN: 0165-5515; 1741-6485
• DOI: 10.1177/0165551515617393

Qualitative studies, such as sociological research, opinion analysis and media studies, can benefit greatly from automated topic mining provided by topic models such as latent Dirichlet allocation (LDA). However, examples of qualitative studies that employ topic modelling as a tool are currently few and far between. In this work, we identify two important problems along the way to using topic models in qualitative studies: lack of a good quality metric that closely matches human judgement in understanding topics and the need to indicate specific subtopics that a specific qualitative study may be most interested in mining. For the first problem, we propose a new quality metric, tf-idf coherence, that reflects human judgement more accurately than regular coherence, and conduct an experiment to verify this claim. For the second problem, we propose an interval semi-supervised approach (ISLDA) where certain predefined sets of keywords (that define the topics researchers are interested in) are restricted to specific intervals of topic assignments. Our experiments show that ISLDA is better for topic extraction than LDA in terms of tf-idf coherence, number of topics identified to predefined keywords and topic stability. We also present a case study on a Russian LiveJournal dataset aimed at ethnicity discourse analysis.