Let's Ask Students About Their Programs, Automatically

TL;DR

This paper proposes an approach to automatically generate questions about student-written code to assess and enhance their understanding, aiming to improve automated assessment systems and promote deeper learning.

Contribution

It introduces a method for automatic question generation about student programs and discusses its potential use in assessment and learning enhancement.

Findings

Questions can reveal students' understanding of code

Automated questions can improve assessment accuracy

Potential to deepen student engagement and learning

Abstract

Students sometimes produce code that works but that its author does not comprehend. For example, a student may apply a poorly-understood code template, stumble upon a working solution through trial and error, or plagiarize. Similarly, passing an automated functional assessment does not guarantee that the student understands their code. One way to tackle these issues is to probe students' comprehension by asking them questions about their own programs. We propose an approach to automatically generate questions about student-written program code. We moreover propose a use case for such questions in the context of automatic assessment systems: after a student's program passes unit tests, the system poses questions to the student about the code. We suggest that these questions can enhance assessment systems, deepen student learning by acting as self-explanation prompts, and provide a window into students' program comprehension. This discussion paper sets an agenda for future technical development and empirical research on the topic.