Ratee Competency Level in Classroom Assessment: Application of Many Facet Rasch Model

Abstract

This study examines teachers’ Classroom Assessment (CA) competency using the Many Facet Rasch Model (MFRM) analysis. Instruments with good psychometric characteristics can guarantee more accurate and fair measurement to respondents. If such instruments are not developed, it is feared that teachers cannot identify their competency level in the CA. The instrument consists of 56 items built based on three primary constructs: knowledge in CA, skills in CA, and attitude towards CA. The research design of this study is a quantitative method with a multi-rater approach using a questionnaire distributed to the raters. Respondents are 262 raters: The Head of Mathematics and Science Department, The Head of Mathematics Panel, and the Mathematics Teacher to assess 100 ratees. The ratees involved in this study are 100 secondary school Mathematics teachers from Selangor. The results show that among the advantages of MFRM are that it can determine the ability and consistency level of the ratees and also detect unexpected responses by the ratees. This study indicates that MFRM is an alternative model suitable to overcome the limitations in Classical Test Theory (CTT) and statistical models in multi-rater analysis. MFRM has the advantage of providing complete information and contributes to understanding the consistent analysis of the ratees’ ability with quantitative evidence support. Furthermore, MFRM can produce better and more precise measurements and make it easier for researchers to communicate the findings.