The relationship between self‐assessment and performance in learning TPACK: Are self‐assessments a good way to support preservice teachers' learning? - Max, Anna‐Lisa; Lukas, Sarah; Weitzel, Holger (Journal of computer assisted learning, 2022)
Background To purposefully integrate information and communication technologies (ICT) into contemporary teaching, teachers must have complex technological, pedagogical and content‐related knowledge (TPACK). TPACK could be fostered through diverse learning environments such as working in a makerspace as part of university teacher training. The development of TPACK in (preservice) teachers depends on how effectively they can assess their own knowledge and adapt their own learning process. Self‐assessment tools can be used to identify and assess their individual TPACK. Objectives The study presents data on TPACK development through project work in a makerspace. It examines the consistency of self‐assessment and performance tests in relation to TPACK and discusses the applicability of self‐assessment scales to self‐regulated learning. Methods TPACK self‐assessment takes place among preservice teacher students (N = 327) in a pre‐post design. This is complemented and contrasted with an external assessment in form of a self‐developed performance test (teaching vignettes). Results Self‐assessment and performance test show a TPACK‐increase resulting in the project work in the makerspace, with higher performing students benefiting more from the intervention than lower performing students. All students tend to overestimate their TPACK before and after the intervention, with low‐performing students overestimating themselves more than high‐performing students. Existing TPACK scales can serve as readily available self‐assessment tools, but should be complemented by other knowledge assessment measures. Furthermore, depending on the results of the self‐assessment, different learning opportunities should be made available so that preservice teachers with a lower TPACK still benefit from the project work in the makerspace. Take Away Self‐assessments accompanying the work in the makerspace could provide feedback information about TPACK‐related knowledge, learning processes and learning outcomes and thus support students in structuring their work. For this purpose, it seems necessary to improve the quality of self‐assessment to enable reliable reflection on preservice teachers' own learning process. Lay Description What is already known about this topic The opportunities for using ICT in schools are developing rapidly, and with them, the demands on teachers' skills (TPACK; e.g., Becker et al., 2020; Eickelmann & Gerick, 2020; Redecker, 2017). Many studies measuring TPACK largely use measurement instruments that required students to self‐assess their TPACK (e.g., Handal et al., 2013; Kaplon‐Schilis & Lyublinskaya, 2019; Lee & Tsai, 2010; Willermark, 2017). Research on TPACK emphasizes that self‐testing TPACK via self‐assessment should be complemented by performance tests in order to better understand the domain‐specific nature of TPACK and its complexity (e.g., Akyuz, 2018; Graham et al., 2009; Voogt et al., 2013). What this paper adds The research provides information on the extent to which TPACK changes triggered by an intervention can be captured using self‐assessment scales and a self‐developed performance test. The data shows how consistent students' self‐assessments are compared to their demonstrated performance. Furthermore, we critically discuss to what extent scales developed to measure TPACK in scientific studies can also be applied to self‐directed learning. Implications for practice Prospective teachers must acquire TPACK competences for the implementation of contemporary teaching, which will enable them to produce digital teaching and learning materials in the future, for example. This TPACK is to be trained through self‐organized project work in the pedagogical Makerspace within the framework of university subject teaching. Self‐assessment instruments for TPACK can be more easily used in the field to accompany projects so that students can receive feedback on knowledge, learning process, and learning outcome. However, our study shows that existing self‐assessment instruments can be used mainly in self‐directed learning processes when learning gains on the complexity levels ‘remembering’ and ‘understanding’ are to be captured, even though feedback especially in the area of ‘applying’ TPACK is of great importance for teaching practice. Combining self‐assessments (based on work samples) and performance tests is therefore a promising approach, as the complementary use of performance tests provides insights into preservice teachers' abilities to ‘apply’ TPACK. The moderate correlation between self‐assessment and performance in our study also shows that self‐assessment is a skill that needs to be learned. Therefore, the use of self‐assessments for self‐regulated learning should be more accompanied by learning opportunities for learning self‐assessment (Bol et al., 2012).