Computational Thinking Development in AI Agent Creation_A Mixed-Methods Study

TL;DR

This study explores how a five-day AI agent creation workshop enhances computational thinking in pre-high school students, revealing an optimal development zone and the importance of tailored scaffolding.

Contribution

It provides empirical evidence on CT development patterns and the impact of initial CT levels, highlighting the need for differentiated instructional strategies.

Findings

Significant improvements in abstract and algorithmic thinking (d ≈ 0.70)

Moderate-CT students showed greater gains than high- or low-CT peers

Engagement in iterative testing predicts self-efficacy gains

Abstract

This mixed-methods study examined computational thinking (CT) development among 93 pre-high school students in a five-day AI agent creation workshop using CocoFlow, a no-code platform. Integrating pre-post assessments, behavioral logs, and interviews, we investigated CT development and how initial CT levels shape learning trajectories. Results revealed significant improvements in abstract thinking (effect size d = 0.71) and algorithmic thinking (effect size d = 0.70). Hierarchical regression identified iterative testing engagement as a predictor of self-efficacy gains (beta = 0.20, p = 0.05). Notably, students with moderate initial CT levels demonstrated substantially greater gains than both high-CT and low-CT peers, revealing an Optimal Development Zone effect (eta squared = 0.55). Qualitative analysis showed moderate-CT students exhibited adaptive expertise, while high-CT students risked over-engineering and low-CT students struggled with task decomposition. These findings challenge linear learning assumptions and provide evidence for differentiated scaffolding in CT education.