HSEmotion Team at ABAW-10 Competition: Facial Expression Recognition, Valence-Arousal Estimation, Action Unit Detection and Fine-Grained Violence Classification

TL;DR

This paper introduces a fast, embedding-based approach for multiple affective behavior analysis tasks in-the-wild, achieving significant improvements over existing baselines in the ABAW challenge.

Contribution

The authors propose a unified, efficient method combining pre-trained models and simple classifiers for facial expression, valence-arousal, action units, and violence detection.

Findings

Significant performance improvements over baselines

Effective noise mitigation via sliding window smoothing

Versatile approach applicable to multiple affective tasks

Abstract

This article presents our results for the 10th Affective Behavior Analysis in-the-Wild (ABAW) competition. For frame-wise facial emotion understanding tasks (frame-wise facial expression recognition, valence-arousal estimation, action unit detection), we propose a fast approach based on facial embedding extraction with pre-trained EfficientNet-based emotion recognition models. If the latter model's confidence exceeds a threshold, its prediction is used. Otherwise, we feed embeddings into a simple multi-layered perceptron trained on the AffWild2 dataset. Estimated class-level scores are smoothed in a sliding window of fixed size to mitigate noise in frame-wise predictions. For the fine-grained violence detection task, we examine several pre-trained architectures for frame embeddings and their aggregation for video classification. Experimental results on four tasks from the ABAW challenge demonstrate that our approach significantly improves validation metrics over existing baselines.