Multiple tutorial-based assessments: a generalizability study

• Published in: BMC medical education Vol. 14; no. 1; p. 30
• Main Authors: St-Onge, Christina, Frenette, Eric, Côté, Daniel J, De Champlain, André
• Format: Journal Article
• Language: English
• Published: England BioMed Central 15.02.2014; BioMed Central Ltd
• Subjects: Academic Achievement; Anatomy; Cooperative Learning; Education; Educational Measurement - methods; Focus Groups; Generalizability Theory; Group dynamics; Independent study; Interpersonal Competence; Learning; Measurement Techniques; Medical Education; Problem Based Learning; Program Evaluation; Psychometrics; Reproducibility of Results; Research and Development; Teaching Methods; Tutors
• ISSN: 1472-6920; 1472-6920
• DOI: 10.1186/1472-6920-14-30

Tutorial-based assessment commonly used in problem-based learning (PBL) is thought to provide information about students which is different from that gathered with traditional assessment strategies such as multiple-choice questions or short-answer questions. Although multiple-observations within units in an undergraduate medical education curriculum foster more reliable scores, that evaluation design is not always practically feasible. Thus, this study investigated the overall reliability of a tutorial-based program of assessment, namely the Tutotest-Lite. More specifically, scores from multiple units were used to profile clinical domains for the first two years of a system-based PBL curriculum. G-Study analysis revealed an acceptable level of generalizability, with g-coefficients of 0.84 and 0.83 for Years 1 and 2, respectively. Interestingly, D-Studies suggested that as few as five observations over one year would yield sufficiently reliable scores. Overall, the results from this study support the use of the Tutotest-Lite to judge clinical domains over different PBL units.