Dictionaries and distributions: Combining expert knowledge and large scale textual data content analysis

• Published in: Behavior research methods Vol. 50; no. 1; p. 344
• Main Authors: Garten, Justin, Hoover, Joe, Johnson, Kate M, Boghrati, Reihane, Iskiwitch, Carol, Dehghani, Morteza
• Format: Journal Article
• Language: English
• Published: New York Springer Nature B.V 01.02.2018
• Subjects: Data analysis; Data processing; Dictionaries; Knowledge; Measurement; Qualitative research
• ISSN: 1554-3528
• DOI: 10.3758/s13428-017-0875-9

Theory-driven text analysis has made extensive use of psychological concept dictionaries, leading to a wide range of important results. These dictionaries have generally been applied through word count methods which have proven to be both simple and effective. In this paper, we introduce Distributed Dictionary Representations (DDR), a method that applies psychological dictionaries using semantic similarity rather than word counts. This allows for the measurement of the similarity between dictionaries and spans of text ranging from complete documents to individual words. We show how DDR enables dictionary authors to place greater emphasis on construct validity without sacrificing linguistic coverage. We further demonstrate the benefits of DDR on two real-world tasks and finally conduct an extensive study of the interaction between dictionary size and task performance. These studies allow us to examine how DDR and word count methods complement one another as tools for applying concept dictionaries and where each is best applied. Finally, we provide references to tools and resources to make this method both available and accessible to a broad psychological audience.