Meta-Analysis of Equivalence of Computerized and P&P Tests on Ability Measures

This study was conducted to investigate the equivalence of scores from paper-and-pencil (P&P) tests and computerized tests (CTs) through meta-analysis of primary studies using both kinds of tests. For this synthesis, 51 primary studies were selected, resulting in 226 effect sizes. The first synt...

Full description

Saved in:
Bibliographic Details
Main Author Kim, Jong-Pil
Format eBook
LanguageEnglish
Published 01.10.1999
Subjects
Adaptive Testing
Computer Assisted Testing
Effect Size
Meta Analysis
Scores
Statistical Significance
Synthesis
Online AccessGet full text

Cover

More Information
Summary:This study was conducted to investigate the equivalence of scores from paper-and-pencil (P&P) tests and computerized tests (CTs) through meta-analysis of primary studies using both kinds of tests. For this synthesis, 51 primary studies were selected, resulting in 226 effect sizes. The first synthesis was a typical meta-analysis that treated multiple measures from the same subjects within studies as independent data. The second synthesis represented results using composite effect sizes. The results from both syntheses were compared in terms of grand mean effect size and the findings for moderator variables. The results of one analysis indicate that eliminating dependence between equivalent scores does not affect the significance of homogeneity tests very much. Overall, ignoring non-independence between equivalent scores tends to lead to underestimated standard errors and inflated Type I error rate when determining statistical significance tests. This is not always true, however, because the means, dispersions, and distributions of equivalent scores depend partly on the number of equivalent scores and partly on the methods for adjusting for dependence of equivalent scores. The type of computerized test was the most important variable when evaluating the equivalence between CT and P&P tests. For computer adapted tests, mathematics, source, and possibly sampling age are significant variables, but for computer based tests, the analyses did not find a significant moderator. (Contains 11 tables, 1 figure, and 78 references.) (SLD)