Selecting the top artisan woodcraft projects using many facet rasch measurement (MFRM) model

Abstract

Once a year, the Geng Seni Kayu Kreatif (GSKK – Creative Woodworkers Group) society conducts a competition for budding craft persons to showcase their ideas and skills. The main objectives were two-fold. First is to check if the selected rubrics are suitable. Second was to rank the woodcraft entries, as fairly as possible. 12 items were identified to measure the woodcraft quality aspects. 20 craft entries were randomly rated by three judges. Rasch MFRM model was used to analyze and rank the items and entries. Item mean is 0.46 and reliability is 0.88. Rater reliability 0.98 indicates raters could reliably separate the entries and items. None of the entries were found misfitting. The scale structure did not exhibit disordered Rasch-Andrich thresholds and had good progressive average measures. Only three entries (E13, E16 and E20) were identified with significant bias among raters, due to significant stricter marks given by one of the raters. Overall, this framework rubric is considered useful for judging woodcraft competitions. It provides non, or very little bias on judgement, and allow less time to judge a competition. However higher-order items need to be improved for future competition, to segregate top entries.