The competent Computational Thinking test (cCTt): a valid, reliable and gender-fair test for longitudinal CT studies in grades 3-6

TL;DR

This study validates the competent CT test (cCTt) as a reliable, gender-fair tool for assessing computational thinking in grades 3-6, facilitating longitudinal studies and transitions between assessments.

Contribution

It introduces a validated, gender-fair, grade-specific CT assessment with proficiency levels and transition scales for longitudinal research.

Findings

cCTt is valid and reliable for grades 3-6

Provides grade-specific proficiency levels

Offers normalized scoring for cross-grade comparison

Abstract

The introduction of computing education into curricula worldwide requires multi-year assessments to evaluate the long-term impact on learning. However, no single Computational Thinking (CT) assessment spans primary school, and no group of CT assessments provides a means of transitioning between instruments. This study therefore investigated whether the competent CT test (cCTt) could evaluate learning reliably from grades 3 to 6 (ages 7-11) using data from 2709 students. The psychometric analysis employed Classical Test Theory, Item Response Theory, Measurement Invariance analyses which include Differential Item Functioning, normalised z-scoring, and PISA's methodology to establish proficiency levels. The findings indicate that the cCTt is valid, reliable and gender-fair for grades 3-6, although more complex items would be beneficial for grades 5-6. Grade-specific proficiency levels are provided to help tailor interventions, with a normalised scoring system to compare students across and between grades, and help establish transitions between instruments. To improve the utility of CT assessments among researchers, educators and practitioners, the findings emphasise the importance of i) developing and validating gender-fair, grade-specific, instruments aligned with students' cognitive maturation, and providing ii) proficiency levels, and iii) equivalency scales to transition between assessments. To conclude, the study provides insight into the design of longitudinal developmentally appropriate assessments and interventions.