Search-based Testing for Scratch Programs

TL;DR

This paper presents a novel search-based method using grammatical evolution to automatically generate test suites for Scratch programs, enhancing feedback and support for learners and educators.

Contribution

It introduces a new approach for test generation in Scratch using grammatical evolution, enabling automation and integration with existing frameworks.

Findings

Effective test suite generation demonstrated on sample Scratch programs

Separation of search encoding from implementation details improves flexibility

Potential for automated feedback in educational settings

Abstract

Block-based programming languages enable young learners to quickly implement fun programs and games. The Scratch programming environment is particularly successful at this, with more than 50 million registered users at the time of this writing. Although Scratch simplifies creating syntactically correct programs, learners and educators nevertheless frequently require feedback and support. Dynamic program analysis could enable automation of this support, but the test suites necessary for dynamic analysis do not usually exist for Scratch programs. It is, however, possible to cast test generation for Scratch as a search problem. In this paper, we introduce an approach for automatically generating test suites for Scratch programs using grammatical evolution. The use of grammatical evolution clearly separates the search encoding from framework-specific implementation details, and allows us to use advanced test acceleration techniques. We implemented our approach as an extension of the Whisker test framework. Evaluation on sample Scratch programs demonstrates the potential of the approach.