Longitudinal study of school climate instrument with secondary school students : validity and reliability analysis with the rasch model

Abstract

The school climate plays a pivotal role in students’ outcomes. Previous literature has highlighted several methodological approaches employed in the school climate domain, including longitudinal studies. However, little is known about the validity and reliability of school climate instruments for longitudinal studies using Rasch analysis. Rasch model is a powerful approach to validate assessment on both item and test levels. Rasch model is coined from the probability of each response and includes item difficulty parameters to characterize the measured items. Moreover, the score represents the item and the person involved with the assessment. Thus, the current study aimed to validate school climate instruments for longitudinal studies with a six-month gap within the context of secondary school students by utilising Rasch analysis. This study evaluated aspects of reliability and validity, such as unidimensionality, rating scale analysis, item fit statistics, item targeting, and differential item functioning. A total of 1,495 secondary school students from public schools in Selangor, Malaysia, completed a 28-item Malay version of the school climate survey at Time-1 and Time-2, with a six-month gap. The results of the Rasch analysis indicated that the instrument had excellent reliability and separation indices, excellent unidimensionality and construct validity, a functional rating scale, good item-person targeting, and good item fit statistics. The current findings provided valid and reliable insights pertinent for policymakers to strategise interventions and initiatives to enhance the quality of school climate and overall education, particularly in the Asian context.