Beyond Scores: Explainable Intelligent Assessment Strengthens Pre-service Teachers' Assessment Literacy

TL;DR

This paper introduces XIA, an explainable assessment platform that enhances pre-service teachers' assessment literacy by providing visualized reasoning and explanations, fostering reflection and reducing errors.

Contribution

It presents a novel intelligent assessment tool with explainable features that bridges assessment theory and practice in teacher education.

Findings

XIA supported reflection and self-regulation.

It reduced assessment errors among pre-service teachers.

Participants shifted toward evidence-based reasoning.

Abstract

Assessment literacy (AL) is essential for personalized education, yet difficult to cultivate in pre-service teachers. Conventional teacher preparation programs focus on theoretical knowledge, while digital assessment tools commonly provide opaque scores or parameters. These limitations hinder reflection and transfer, leaving AL underdeveloped. We propose XIA, an eXplainable Intelligent Assessment platform that extends statistics-informed support with visualized cognitive diagnostic reasoning, including contrastive and counterfactual explanations. In a pre-post controlled study with 21 pre-service teachers, we combined quantitative tasks and questionnaires with qualitative interviews. The findings offer preliminary evidence that XIA supported reflection, self-regulation, and assessment awareness, and helped reduce assessment errors. Interviews further showed a shift from score-based judgments toward evidence-based reasoning. This work contributes insights into the design of intelligent assessment tools, showing how explanatory scaffolding can bridge assessment theory and classroom practice and support the cultivation of AL in teacher education.