Oversight in Action: Experiences with Instructor-Moderated LLM Responses in an Online Discussion Forum
Shuying Qiao, Paul Denny, Nasser Giacaman

TL;DR
This paper describes the design, deployment, and evaluation of an instructor-controlled bot in an online forum that drafts responses to student questions, aiming to reduce instructor workload while maintaining oversight of LLM-generated content.
Contribution
It introduces a novel instructor-moderated bot system integrated into online discussion forums to assist with managing LLM responses in computing education.
Findings
Alleviated instructor workload effectively.
Facilitated anonymous student participation.
Identified challenges with complex queries.
Abstract
The integration of large language models (LLMs) into computing education offers many potential benefits to student learning, and several novel pedagogical approaches have been reported in the literature. However LLMs also present challenges, one of the most commonly cited being that of student over-reliance. This challenge is compounded by the fact that LLMs are always available to provide instant help and solutions to students, which can undermine their ability to independently solve problems and diagnose and resolve errors. Providing instructor oversight of LLM-generated content can mitigate this problem, however it is often not practical in real-time learning contexts. Online class discussion forums, which are widely used in computing education, present an opportunity for exploring instructor oversight because they operate asynchronously. Unlike real-time interactions, the discussion…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsOnline and Blended Learning
