Rater Severity in Measuring Teachers’ Competency in Classroom Assessment

Abstract

Many Facet Rasch Model (MFRM) provided more honest information on the validity of the judgement process by considering the factors of raters’ variability and estimating the ratee abilities without depending on the severity or leniency of raters. The multi-rater method produced a more stable and precise assessment and has higher reliability than the self- assessment method. However, the raters have different characteristics that can influence their severity level when evaluating the ratees. This study aims to investigate the effect of gender and job position on rater severity when assessing teachers’ competency in Classroom Assessment (CA) by using MFRM. The instrument consists of 56 items built based on three main constructs; knowledge in CA, skills in CA, and attitude towards CA. This study used a quantitative multi- rater approach using a questionnaire distributed to 262 raters to assess 100 ratees. Raters’ severity levels form four categories based on the logit measure value. There are no significant mean differences between the rating of male and female raters. Similarly, there is no significant mean difference between the rating based on job positions held by the raters.