Enhanced Classroom Dialogue Sequences Analysis with a Hybrid AI Agent: Merging Expert Rule-Base with Large Language Models

TL;DR

This paper introduces a hybrid AI system that combines expert rules with large language models to improve the analysis of classroom dialogue sequences, making it more accurate, scalable, and adaptable.

Contribution

It develops a novel AI agent integrating rule-based and LLM approaches for classroom dialogue analysis, bridging theoretical and empirical methods.

Findings

High precision and reliability in dialogue categorisation

Enhanced efficiency and scalability of analysis

Potential to improve teaching practices and teacher development

Abstract

Classroom dialogue plays a crucial role in fostering student engagement and deeper learning. However, analysing dialogue sequences has traditionally relied on either theoretical frameworks or empirical descriptions of practice, with limited integration between the two. This study addresses this gap by developing a comprehensive rule base of dialogue sequences and an Artificial Intelligence (AI) agent that combines expert-informed rule-based systems with a large language model (LLM). The agent applies expert knowledge while adapting to the complexities of natural language, enabling accurate and flexible categorisation of classroom dialogue sequences. By synthesising findings from over 30 studies, we established a comprehensive framework for dialogue analysis. The agent was validated against human expert coding, achieving high levels of precision and reliability. The results demonstrate that the agent provides theory-grounded and adaptive functions, tremendously enhancing the efficiency and scalability of classroom dialogue analysis, offering significant potential in improving classroom teaching practices and supporting teacher professional development.