Designing the virtual CAT: A digital tool for algorithmic thinking assessment in compulsory education

TL;DR

This paper introduces the virtual Cross Array Task (CAT), a digital assessment tool designed to evaluate algorithmic thinking skills in Swiss compulsory education through iterative design, participatory feedback, and pilot testing across diverse student groups.

Contribution

It presents the design, development, and initial evaluation of a scalable, accessible digital assessment platform for algorithmic thinking in early education, incorporating multilingual and multimodal interfaces.

Findings

The platform is usable and accessible for diverse learners.

Preliminary results indicate effective assessment of AT skills across age groups.

The tool is adaptable to various educational contexts.

Abstract

Algorithmic thinking (AT) is a critical skill in today's digital society, and it is indispensable not only in computer science-related fields but also in everyday problem-solving. As a foundational component of digital education and literacy, fostering AT skills is increasingly relevant for all students and should become a standard part of compulsory education. However, successfully integrating AT into formal education requires effective teaching strategies and robust and scalable assessment procedures. In this paper, we present the design and development process of the virtual Cross Array Task (CAT), a digital adaptation of an unplugged assessment activity aimed at evaluating algorithmic skills in Swiss compulsory education. The development process followed iterative design cycles, incorporating expert evaluations to refine the tool's usability, accessibility and functionality. A participatory design study played a dual role in shaping the platform. First, it gathered valuable insights from end users, including students and teachers, to ensure the tool's relevance and practicality in classroom settings. Second, it facilitated the collection and preliminary analysis of data related to students' AT skills, providing an initial evaluation of the tool's assessment capabilities across various developmental stages. This was achieved through a pilot study involving a diverse group of students aged 4 to 12, spanning preschool to lower secondary school levels. The resulting instrument features multilingual support and includes both gesture-based and visual block-based programming interfaces, making it accessible to a broad range of learners. Findings from the pilot study demonstrate the platform's usability and accessibility, as well as its suitability for assessing AT skills, with preliminary results showing its ability to cater to diverse age groups and educational contexts.