Automatic Content Analysis of Online Discussions for Cognitive Presence: A Study of the Generalizability Across Educational Contexts

• Published in: IEEE transactions on learning technologies Vol. 14; no. 3; pp. 299 - 312
• Main Authors: Neto, Valter, Rolim, Vitor, Pinheiro, Anderson, Lins, Rafael Dueire, Gasevic, Dragan, Mello, Rafael Ferreira
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.06.2021; Institute of Electrical and Electronics Engineers, Inc; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Analytical models; Automation; Biology; Classification; Classifiers; Cognitive style; Communities of Practice; Community of inquiry model (CoI); Computer Mediated Communication; Content Analysis; context analysis; Context modeling; Data mining; Datasets; Discussion; Education; Educational tests & measurements; Feature extraction; Foreign Countries; Identification; learning analytics; Manuals; Messages; Online Courses; online discussion; Portuguese; Reliability; Teacher Role; Technology; Testing; Text mining; Brazil
• ISSN: 1939-1382; 2372-0050
• DOI: 10.1109/TLT.2021.3083178

This article investigates the impact of educational contexts on automatic classification of online discussion messages according to cognitive presence, an essential construct of the community of inquiry model. In particular, the work reported in the article analyzed online discussion messages written in Brazilian Portuguese from two different courses that were from different subject areas (biology and technology) and had different teaching presence in the online discussions. The study explored a set of 127 features of online discussion messages and a random forest classifier to automatically recognize the phases of the cognitive presence in online discussion messages. The results showed that the classifier achieved better performance when applied to the entire dataset. It reveals that when a classifier is created for a specific course it is not generic enough to be applied to a course from a different field of knowledge. The results also showed the importance of the features that were predictive of the phases of the cognitive presence in the educational context. Based on the findings of this study, future work should adopt the same feature set as used in the current study, but it should train the classifier of the cognitive presence on datasets in subject areas related to the topic of the discussions.