Listening with Language Models: Using LLMs to Collect and Interpret Classroom Feedback

TL;DR

This paper demonstrates how LLM-powered chatbots can transform classroom feedback by enabling real-time, conversational, and detailed student-instructor interactions, leading to richer insights and more adaptive teaching.

Contribution

The paper introduces a novel LLM-based feedback system with three components, showing its effectiveness in collecting and interpreting classroom feedback in higher education.

Findings

LLM feedback systems provide richer insights than traditional surveys.

Students engage more with conversational feedback formats.

Instructors can make mid-course adjustments based on detailed feedback.

Abstract

Traditional end-of-quarter surveys often fail to provide instructors with timely, detailed, and actionable feedback about their teaching. In this paper, we explore how Large Language Model (LLM)-powered chatbots can reimagine the classroom feedback process by engaging students in reflective, conversational dialogues. Through the design and deployment of a three-part system-PromptDesigner, FeedbackCollector, and FeedbackAnalyzer-we conducted a pilot study across two graduate courses at UC Santa Cruz. Our findings suggest that LLM-based feedback systems offer richer insights, greater contextual relevance, and higher engagement compared to standard survey tools. Instructors valued the system's adaptability, specificity, and ability to support mid-course adjustments, while students appreciated the conversational format and opportunity for elaboration. We conclude by discussing the design implications of using AI to facilitate more meaningful and responsive feedback in higher education.