From Misunderstandings to Learning Opportunities: Leveraging Generative AI in Discussion Forums to Support Student Learning

TL;DR

This paper presents M2M, a novel approach using LLMs and RAG to identify student misunderstandings in discussion forums, helping instructors improve teaching and student learning in large classes.

Contribution

The paper introduces M2M, a new method leveraging LLMs and RAG to detect misunderstandings in student discussions, validated with real data from multiple courses.

Findings

Instructors found M2M effective for identifying misunderstandings.

The approach generated actionable insights for teaching improvement.

Feedback highlighted the need for finer groupings and ethical considerations.

Abstract

In the contemporary educational landscape, particularly in large classroom settings, discussion forums have become a crucial tool for promoting interaction and addressing student queries. These forums foster a collaborative learning environment where students engage with both the teaching team and their peers. However, the sheer volume of content generated in these forums poses two significant interconnected challenges: How can we effectively identify common misunderstandings that arise in student discussions? And once identified, how can instructors use these insights to address them effectively? This paper explores the approach to integrating large language models (LLMs) and Retrieval-Augmented Generation (RAG) to tackle these challenges. We then demonstrate the approach Misunderstanding to Mastery (M2M) with authentic data from three computer science courses, involving 1355 students with 2878 unique posts, followed by an evaluation with five instructors teaching these courses. Results show that instructors found the approach promising and valuable for teaching, effectively identifying misunderstandings and generating actionable insights. Instructors highlighted the need for more fine-grained groupings, clearer metrics, validation of the created resources, and ethical considerations around data anonymity.