Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Application

TL;DR

This paper introduces TalkMoves, a transformer-based application that analyzes classroom recordings to give teachers personalized feedback on discourse strategies, aiming to improve mathematics instruction and student engagement.

Contribution

The paper presents a novel transformer-based model and a cloud infrastructure for analyzing classroom speech and providing targeted feedback to teachers.

Findings

Achieved a model F1 score of 79.3% in identifying talk moves.

Developed a cloud-based system for processing noisy classroom audio.

Addressed technical challenges in real-world speech data analysis.

Abstract

TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application combines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types of discourse aimed at broadening and deepening classroom conversations about mathematics. These specific discourse strategies are referred to as "talk moves" within the mathematics education community and prior research has documented the ways in which systematic use of these discourse strategies can positively impact student engagement and learning. In this article, we describe the TalkMoves application's cloud-based infrastructure for managing and processing classroom recordings, and its interface for providing teachers with feedback on their use of talk moves during individual teaching episodes. We present the series of model architectures we developed, and the studies we conducted, to develop our best-performing, transformer-based model (F1 = 79.3%). We also discuss several technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms.