Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification

TL;DR

This paper presents a 3D-CNN based micro-gesture recognition method using skeletal and semantic embedding loss, achieving first place in the IJCAI 2023 Micro-gesture Classification Challenge.

Contribution

It introduces a novel skeletal and semantic embedding loss for micro-gesture classification with 3D-CNNs, improving accuracy over existing methods.

Findings

Achieved 1st place in the challenge with top accuracy

Surpassed second-place team by 1.10% in Top-1 accuracy

Demonstrated effectiveness of embedding loss in gesture recognition

Abstract

In this paper, we briefly introduce the solution of our team HFUT-VUT for the Micros-gesture Classification in the MiGA challenge at IJCAI 2023. The micro-gesture classification task aims at recognizing the action category of a given video based on the skeleton data. For this task, we propose a 3D-CNNs-based micro-gesture recognition network, which incorporates a skeletal and semantic embedding loss to improve action classification performance. Finally, we rank 1st in the Micro-gesture Classification Challenge, surpassing the second-place team in terms of Top-1 accuracy by 1.10%.