Towards Student Actions in Classroom Scenes: New Dataset and Baseline

TL;DR

This paper introduces a new dataset of classroom student actions and a baseline transformer-based method for action detection, aiming to advance AI tools for educational research and classroom analysis.

Contribution

The paper presents the SAV dataset with 4,324 annotated classroom videos and proposes a novel visual transformer baseline for improved action detection in complex classroom scenes.

Findings

The SAV dataset includes 15 student actions across diverse classroom scenarios.

The proposed transformer-based method achieves a mAP of 67.9% on SAV.

The dataset presents challenges like subtle movements, dense objects, and occlusions.

Abstract

Analyzing student actions is an important and challenging task in educational research. Existing efforts have been hampered by the lack of accessible datasets to capture the nuanced action dynamics in classrooms. In this paper, we present a new multi-label Student Action Video (SAV) dataset, specifically designed for action detection in classroom settings. The SAV dataset consists of 4,324 carefully trimmed video clips from 758 different classrooms, annotated with 15 distinct student actions. Compared to existing action detection datasets, the SAV dataset stands out by providing a wide range of real classroom scenarios, high-quality video data, and unique challenges, including subtle movement differences, dense object engagement, significant scale differences, varied shooting angles, and visual occlusion. These complexities introduce new opportunities and challenges to advance action detection methods. To benchmark this, we propose a novel baseline method based on a visual transformer, designed to enhance attention to key local details within small and dense object regions. Our method demonstrates excellent performance with a mean Average Precision (mAP) of 67.9% and 27.4% on the SAV and AVA datasets, respectively. This paper not only provides the dataset but also calls for further research into AI-driven educational tools that may transform teaching methodologies and learning outcomes. The code and dataset are released at https://github.com/Ritatanz/SAV.