Using LLMs to support assessment of student work in higher education: a viva voce simulator

TL;DR

This paper introduces a novel LLM-based viva voce simulator designed to assist in assessing student work by generating interactive questions and evaluating student responses to determine authorship.

Contribution

The paper presents a new LLM-powered tool for supporting assessment of written student work through simulated oral examinations, addressing challenges posed by AI-generated submissions.

Findings

The simulator can generate plausible questions for student assessment.

It provides interaction transcripts to assist human examiners.

The approach offers a practical method for verifying student authorship.

Abstract

One of the emergent challenges of student work submitted for assessment is the widespread use of large language models (LLMs) to support and even produce written work. This particularly affects subjects where long-form written work is a key part of assessment. We propose a novel approach to addressing this challenge, using LLMs themselves to support the assessment process. We have developed a proof-of-concept viva voce examination simulator, which accepts the student's written submission as input, generates an interactive series of questions from the LLM and answers from the student. The viva voce simulator is an interactive tool which asks questions which a human examiner might plausibly ask, and uses the student's answers to form a judgment about whether the submitted piece of work is likely to be the student's own work. The interaction transcript is provided to the human examiner to support their final judgment. We suggest theoretical and practical points which are critical to real-world deployment of such a tool.