Student Classroom Behavior Detection based on YOLOv7-BRA and Multi-Model Fusion

TL;DR

This paper presents a novel system for detecting student classroom behaviors using an enhanced YOLOv7 model with attention mechanisms and multi-model fusion, achieving high accuracy on a custom dataset.

Contribution

The study introduces YOLOv7-BRA with Bi-level Routing Attention and a multi-model fusion approach, improving behavior detection accuracy in classroom videos.

Findings

Achieved 87.1% mAP@0.5 on SCB-Dataset

Improved detection accuracy by 2.2% over previous methods

Constructed a new dataset with 11,248 labels and 4,001 images

Abstract

Accurately detecting student behavior in classroom videos can aid in analyzing their classroom performance and improving teaching effectiveness. However, the current accuracy rate in behavior detection is low. To address this challenge, we propose the Student Classroom Behavior Detection system based on based on YOLOv7-BRA (YOLOv7 with Bi-level Routing Attention ). We identified eight different behavior patterns, including standing, sitting, speaking, listening, walking, raising hands, reading, and writing. We constructed a dataset, which contained 11,248 labels and 4,001 images, with an emphasis on the common behavior of raising hands in a classroom setting (Student Classroom Behavior dataset, SCB-Dataset). To improve detection accuracy, we added the biformer attention module to the YOLOv7 network. Finally, we fused the results from YOLOv7 CrowdHuman, SlowFast, and DeepSort models to obtain student classroom behavior data. We conducted experiments on the SCB-Dataset, and YOLOv7-BRA achieved an mAP@0.5 of 87.1%, resulting in a 2.2% improvement over previous results. Our SCB-dataset can be downloaded from: https://github.com/Whiffe/SCB-datase