A validity study for Yes/No Angoff standard setting method using cluster analysis

Test validity is a property of the interpretation assigned to test scores. To provide an objective validating evidence for a standard-referenced assessment is especially important. In this study we utilize a statistical technique, cluster analysis, to explore the validity of one of the expert judgem...

Full description

Saved in:
Bibliographic Details
Published in2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD) pp. 727 - 731
Main Authors Fen-Lan Tseng, Jia-Min Chiou, Yao-Ting Sung
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.08.2015
Subjects
cluster analysis
Data models
Gaussian distribution
hierarchical clustering
mixture of Gaussian models
Psychology
Qualifications
Reliability
standard setting
Testing
Online AccessGet full text

Cover

More Information
Summary:Test validity is a property of the interpretation assigned to test scores. To provide an objective validating evidence for a standard-referenced assessment is especially important. In this study we utilize a statistical technique, cluster analysis, to explore the validity of one of the expert judgement technique- Yes/No Angoff standard setting method. We first segregated each examinee ability cluster using the hierarchical clustering (HC). Assume that each ability cluster is a Gaussian distribution and that the distribution of each test subject data can be modeled by mixture of Gaussians (MoG), where the mean, variance and the proportion of each cluster were initialized by the HC results. Finally, the ability clustering was implemented by the expectation maximization (EM) method. The results from the traditional standard-setting procedure and cluster analysis were compared. The study suggested that cluster analysis could be applied as a support tool to provide validating information in the process of standard setting for high-stakes achievement tests.
DOI:10.1109/FSKD.2015.7382032