A Spatio-Temporal Attention-Based Method for Detecting Student Classroom Behaviors

TL;DR

This paper introduces a spatio-temporal attention-based method, BDSTA, that enhances student classroom behavior detection accuracy by integrating motion features and addressing data imbalance, achieving significant improvements over existing models.

Contribution

The paper presents a novel BDSTA method combining SlowFast features with spatio-temporal attention and an improved focal loss to improve behavior detection accuracy in classroom videos.

Findings

8.94% increase in classification accuracy over SlowFast

Effective handling of long-tail data with improved focal loss

Successful application on the self-made STSCB dataset

Abstract

Accurately detecting student behavior from classroom videos is beneficial for analyzing their classroom status and improving teaching efficiency. However, low accuracy in student classroom behavior detection is a prevalent issue. To address this issue, we propose a Spatio-Temporal Attention-Based Method for Detecting Student Classroom Behaviors (BDSTA). Firstly, the SlowFast network is used to generate motion and environmental information feature maps from the video. Then, the spatio-temporal attention module is applied to the feature maps, including information aggregation, compression and stimulation processes. Subsequently, attention maps in the time, channel and space dimensions are obtained, and multi-label behavior classification is performed based on these attention maps. To solve the long-tail data problem that exists in student classroom behavior datasets, we use an improved focal loss function to assign more weight to the tail class data during training. Experimental results are conducted on a self-made student classroom behavior dataset named STSCB. Compared with the SlowFast model, the average accuracy of student behavior classification detection improves by 8.94\% using BDSTA.