Comprehensive evaluation criteria for English learning websites using expert validity surveys

• Published in: Computers and education Vol. 51; no. 1; pp. 403 - 422
• Main Authors: Yang, Ya-Ting C., Chan, Chia-Ying
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.08.2008; Elsevier
• Subjects: Content Validity; Discussion Groups; Distributed learning environments; English Instruction; English Teachers; Evaluation Criteria; Human–computer interface; Interactive learning environments; Interrater Reliability; Intonation; Language Skills; Second Language Learning; Speech Communication; Surveys; Teaching/learning strategies; Web Sites; Writing Skills; Distributed learning environments; Teaching/learning strategies; Interactive learning environments; Human–computer interface
• ISSN: 0360-1315; 1873-782X
• DOI: 10.1016/j.compedu.2007.05.011

This study aimed to develop a set of evaluation criteria for English learning websites. These criteria can assist English teachers/web designers in designing effective websites for their English courses and can also guide English learners in screening for appropriate and reliable websites to use in increasing their English ability. To fulfill our objective, we employed a three-phase research procedure: (a) establishing a preliminary set of criteria from a thorough review of the literature, (b) evaluating and refining the preliminary criteria by conducting interviews with in-service teachers and learners, and (c) validating and finalizing the criteria according to expert validity surveys. The established criteria have 46 items, classified into 6 categories (the number of items within the category) – general information (12), integrated English learning (13), listening (4), speaking (6), reading (5), and writing (6). The general information evaluates the authority, accuracy, and format of the learning websites. The integrated English learning evaluates the overall information relevant to English learning materials as well as the common features of the four language skills. The criteria for listening, speaking, reading, and writing, for example, examine the suitable intonation, skills of discourse, classification of reading articles by their attributes, and the proper use of discussion boards for students when practicing their writing skills. Based on qualitative and quantitative analysis of the interviews and expert validity surveys, we confirmed the effectiveness of the developed evaluation criteria with satisfactory indexes of inter-rater reliability, content validity, and factorial validity.