SnapCheck: Automated Testing for Snap Programs

TL;DR

SnapCheck is an automated testing framework for Snap programs that enables high-accuracy, property-based assessment of student projects, facilitating auto-grading and real-time feedback in visual programming environments.

Contribution

The paper introduces SnapCheck, a novel dynamic testing framework with Condition-Action templates for automated, accurate assessment of Snap programs.

Findings

Achieves at least 98% accuracy in grading Snap projects

Effective for auto-grading and formative feedback in educational settings

Demonstrated on 162 code snapshots from a Pong assignment

Abstract

Programming environments such as Snap, Scratch, and Processing engage learners by allowing them to create programming artifacts such as apps and games, with visual and interactive output. Learning programming with such a media-focused context has been shown to increase retention and success rate. However, assessing these visual, interactive projects requires time and laborious manual effort, and it is therefore difficult to offer automated or real-time feedback to students as they work. In this paper, we introduce SnapCheck, a dynamic testing framework for Snap that enables instructors to author test cases with Condition-Action templates. The goal of SnapCheck is to allow instructors or researchers to author property-based test cases that can automatically assess students' interactive programs with high accuracy. Our evaluation of SnapCheck on 162 code snapshots from a Pong game assignment in an introductory programming course shows that our automated testing framework achieves at least 98% accuracy over all rubric items, showing potentials to use SnapCheck for auto-grading and providing formative feedback to students.