Turning Language Model Training from Black Box into a Sandbox

TL;DR

This paper introduces a browser-based tool that makes training small language models transparent, helping students understand AI mechanics through hands-on experience, which improves their conceptual grasp of model behavior.

Contribution

The paper presents a novel educational tool that visualizes language model training on personal devices, enhancing AI literacy by shifting student understanding from anthropomorphic views to data-driven reasoning.

Findings

Students' explanations shifted from anthropomorphic to data-based reasoning.

Hands-on training improved students' understanding of model training.

The tool supports AI literacy in K-12 and higher education contexts.

Abstract

Most classroom engagements with generative AI focus on prompting pre-trained models, leaving the role of training data and model mechanics opaque. We developed a browser-based tool that allows students to train a small transformer language model entirely on their own device, making the training process visible. In a CS1 course, 162 students completed pre- and post-test explanations of why language models sometimes produce incorrect or strange output. After a brief hands-on training activity, students' explanations shifted significantly from anthropomorphic and misconceived accounts toward data- and model-based reasoning. The results suggest that enabling learners to directly observe training can support conceptual understanding of the data-driven nature of language models and model training, even within a short intervention. For K-12 AI literacy and AI education research, the study findings suggest that enabling students to train - and not only prompt - language models can shift how they think about AI.