Effective Feedback for Introductory CS Theory: A JFLAP Extension and Student Persistence

TL;DR

This research introduces a JFLAP extension that provides targeted feedback to help undergraduate students better understand automata and formal languages, leading to improved learning outcomes in introductory CS theory.

Contribution

We developed a JFLAP server extension that offers automated correctness evaluation and witness feedback for various automata models, enhancing student learning in CS theory courses.

Findings

Students using the extension performed better on targeted homework questions.

The extension encouraged persistence and iterative learning among students.

Witness feedback helped students understand concepts more deeply.

Abstract

Computing theory analyzes abstract computational models to rigorously study the computational difficulty of various problems. Introductory computing theory can be challenging for undergraduate students, and the main goal of our research is to help students learn these computational models. The most common pedagogical tool for interacting with these models is the Java Formal Languages and Automata Package (JFLAP). We developed a JFLAP server extension, which accepts homework submissions from students, evaluates the submission as correct or incorrect, and provides a witness string when the submission is incorrect. Our extension currently provides witness feedback for deterministic finite automata, nondeterministic finite automata, regular expressions, context-free grammars, and pushdown automata. In Fall 2019, we ran a preliminary investigation on two sections (Control and Study) of the required undergraduate course Introduction to Computer Science Theory. The Study section used our extension for five targeted homework questions, and the Control section solved and submitted these problems using traditional means. Our results show that on these five questions, the Study section performed better on average than the Control section. Moreover, the Study section persisted in submitting attempts until correct, and from this finding, our preliminary conclusion is that minimal (not detailed or grade-based) witness feedback helps students to truly learn the concepts. We describe the results that support this conclusion as well as a related hypothesis conjecturing that with witness feedback and unlimited number of submissions, partial credit is both unnecessary and ineffective.