Development of an item bank for assessing generic competences in a higher-education institute: a Rasch modelling approach

• Published in: Higher education research and development Vol. 33; no. 4; pp. 821 - 835
• Main Authors: Xie, Qin, Zhong, Xiaoling, Wang, Wen-Chung, Lim, Cher Ping
• Format: Journal Article
• Language: English
• Published: Routledge 01.01.2014
• Subjects: Achievement Tests; Case Studies; Communication skills; Competence; Construct Validity; Content Validity; Correlation; Creative thinking; Critical thinking; Decision making; Focus Groups; Foreign Countries; Generic skills; Higher education; Hong Kong; instrument development; Interpersonal relationship; Item Banks; Item Response Theory; Problem solving; Rasch model; Rasch modelling; Self Evaluation (Individuals); Test construction; Test Reliability; Test Validity; Undergraduate Students; Hong Kong
• ISSN: 0729-4360; 1469-8366
• DOI: 10.1080/07294360.2013.863847

This paper describes the development and validation of an item bank designed for students to assess their own achievements across an undergraduate-degree programme in seven generic competences (i.e., problem-solving skills, critical-thinking skills, creative-thinking skills, ethical decision-making skills, effective communication skills, social interaction skills and global perspective). The Rasch modelling approach was adopted for instrument development and validation. A total of 425 items were developed. The content validity of these items was examined via six focus group interviews with target students, and the construct validity was verified against data collected from a large student sample (N = 1151). A matrix design was adopted to assemble the items in 26 test forms, which were distributed at random in each administration session. The results demonstrated that the item bank had high reliability and good construct validity. Cross-sectional comparisons of Years 1-4 students revealed patterns of changes over the years. Correlation analyses shed light on the relationships between the constructs. Implications are drawn to inform future efforts to develop the instrument, and suggestions are made regarding ways to use the instrument to enhance the teaching and learning of generic skills.