AutoMCQ -- Automatically Generate Code Comprehension Questions using GenAI

TL;DR

AutoMCQ leverages GenAI to automatically generate multiple-choice questions for code comprehension, aiming to improve understanding assessment and scalability in educational settings.

Contribution

The paper presents AutoMCQ, a novel system that automates code comprehension question generation using GenAI, integrated with an existing assessment platform.

Findings

Effective automatic question generation demonstrated

Potential to scale code comprehension assessments

Enhanced detection of misunderstandings and plagiarism

Abstract

Students often do not fully understand the code they have written. This sometimes does not become evident until later in their education, which can mean it is harder to fix their incorrect knowledge or misunderstandings. In addition, being able to fully understand code is increasingly important in a world where students have access to generative artificial intelligence (GenAI) tools, such as GitHub Copilot. One effective solution is to utilise code comprehension questions, where a marker asks questions about a submission to gauge understanding, this can also have the side effect of helping to detect plagiarism. However, this approach is time consuming and can be difficult and/or expensive to scale. This paper introduces AutoMCQ, which uses GenAI for the automatic generation of multiple-choice code comprehension questions. This is integrated with the CodeRunner automated assessment platform.