SPARK: Real-Time Monitoring of Multi-Faceted Programming Exercises

TL;DR

SPARK is a real-time monitoring dashboard designed to help instructors track and analyze student progress in complex, multi-step programming exercises, providing visualizations and insights to improve teaching effectiveness.

Contribution

This paper introduces SPARK, a novel dashboard that enables flexible grouping, automated testing, and detailed visualization for monitoring complex programming exercises.

Findings

SPARK improves instructors' ability to identify student difficulties.

The dataset reveals diverse solution approaches among students.

Instructors find SPARK useful for understanding student progress.

Abstract

Monitoring in-class programming exercises can help instructors identify struggling students and common challenges. However, understanding students' progress can be prohibitively difficult, particularly for multi-faceted problems that include multiple steps with complex interdependencies, have no predictable completion order, or involve evaluation criteria that are difficult to summarize across many students (e.g., exercises building interactive web-based user interfaces). We introduce SPARK, a coding exercise monitoring dashboard designed to address these challenges. SPARK allows instructors to flexibly group substeps into checkpoints based on exercise requirements, suggests automated tests for these checkpoints, and generates visualizations to track progress across steps. SPARK also allows instructors to inspect intermediate outputs, providing deeper insights into solution variations. We also construct a dataset of 40-minute keystroke coding data from N=22 learners solving two web programming exercises and provide empirical insights into the perceived usefulness of SPARK through a within-subjects evaluation with 16 programming instructors.