Abstract
This study examines the psychometric validation of a dichotomous instrument in measuring the critical thinking skills of madrasah tsanawiyah students using Classical Test Theory (CTT) and Item Response Theory (IRT) model 2PL. Responding to the limitations of response category analysis, this study uses descriptive quantitative methodology involving 300 students from MTs NU Ungaran Semarang, Indonesia. Data were collected through a 30-item dichotomous instrument that was content validated (Aiken's V coefficient = 0.89) and analyzed using two IRT models: IRT. The main findings show that the research instrument consists of a 30-item multiple-choice test that measures critical thinking skills regarding the concept of the creation of the universe based on verses from the Qur'an. The CTT analysis showed that the test reliability was 0.727, which is considered quite good. From the initial 30 items, item selection was carried out based on the criteria of item discrimination and difficulty, resulting in 12 selected items with good psychometric quality. IRT 2PL analysis revealed that all items fit the model (p > 0.05) with a range of discrimination parameters (a) between 0.591 and 2.400 and difficulty parameters (b) between -3.265 and 1.632. Reliability based on the IRT model reached 0.802. Items But6, But5, and But2 were items with excellent discrimination (a > 1.6), while But9 and But12 were items with the highest level of difficulty. The 12-item test instrument for understanding the concept of the creation of the universe has proven to have good psychometric quality and is reliable and valid for measuring MTs students' understanding of the concept of the creation of the universe from the perspective of the Qur'an. These findings encourage the adoption of critical thinking assessment techniques in the context of Islamic Religious Education to improve the validity of competency evaluation, with recommendations for further research to explore this topic further. These findings can also be used as an effective assessment tool in the evaluation of Islamic religious education learning.