A qualitative evaluation of evolution of a learning analytics tool

• Published in: Computers and education Vol. 58; no. 1; pp. 470 - 489
• Main Authors: Ali, Liaqat, Hatala, Marek, Gašević, Dragan, Jovanović, Jelena
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 2012; Elsevier
• Subjects: Comparative Analysis; Computer Graphics; Computer Interfaces; Computer Software; Computer Software Evaluation; Computer Uses in Education; Content analysis; Data; Data visualization; Education; Educational Technology; Evaluation Methods; Evaluation Research; Evaluation Utilization; Evolution; Feedback; Feedback (Response); Internet; Learning; Learning Activities; Learning analytics; Mathematical analysis; Program Descriptions; Qualitative evaluation; Qualitative Research; Semantic web; Stemming; Student Evaluation; Use Studies; Visual Aids; Visualization; Semantic web; Visualization; Qualitative evaluation; Learning analytics; Content analysis
• ISSN: 0360-1315; 1873-782X
• DOI: 10.1016/j.compedu.2011.08.030

LOCO-Analyst is a learning analytics tool we developed to provide educators with feedback on students learning activities and performance. Evaluation of the first version of the tool led to the enhancement of the tool’s data visualization, user interface, and supported feedback types. The second evaluation of the improved tool allowed us to see how the improvements affected the users’ perceived value of the tool. Here, we present the qualitative results of our two evaluations and discuss important lessons learned stemming from the comparison of the two studies. The results show that educators find the kinds of feedback implemented in the tool informative and they value the mix of textual and graphical representations of different kinds of feedback provided by the tool. ► a learning analytics tool for educators-directed feedback is presented. ► results of two qualitative evaluation studies of the tool are presented. ► educators found the feedback implemented in the tool informative. ► educators valued the mix of textual and graphical representations of the feedback. ► important lessons learned from the comparison of the two studies are discussed.