Online Micro-gesture Recognition Using Data Augmentation and Spatial-Temporal Attention

TL;DR

This paper presents HFUT-VUT, a novel approach for online micro-gesture recognition that uses data augmentation and spatial-temporal attention, achieving state-of-the-art results in the IJCAI 2025 MiGA Challenge.

Contribution

The paper introduces a new method combining data augmentation and spatial-temporal attention for micro-gesture recognition, surpassing previous methods in accuracy.

Findings

Achieved an F1 score of 38.03, outperforming previous state-of-the-art by 37.9%.

Ranked first in the Micro-gesture Online Recognition track of the challenge.

Demonstrated effectiveness of data augmentation and attention mechanisms in micro-gesture detection.

Abstract

In this paper, we introduce the latest solution developed by our team, HFUT-VUT, for the Micro-gesture Online Recognition track of the IJCAI 2025 MiGA Challenge. The Micro-gesture Online Recognition task is a highly challenging problem that aims to locate the temporal positions and recognize the categories of multiple micro-gesture instances in untrimmed videos. Compared to traditional temporal action detection, this task places greater emphasis on distinguishing between micro-gesture categories and precisely identifying the start and end times of each instance. Moreover, micro-gestures are typically spontaneous human actions, with greater differences than those found in other human actions. To address these challenges, we propose hand-crafted data augmentation and spatial-temporal attention to enhance the model's ability to classify and localize micro-gestures more accurately. Our solution achieved an F1 score of 38.03, outperforming the previous state-of-the-art by 37.9%. As a result, our method ranked first in the Micro-gesture Online Recognition track.