Identifying Ethical Biases in Action Recognition Models

TL;DR

This paper presents a framework for auditing bias in Human Action Recognition models using synthetic video data with controlled visual attributes, revealing potential fairness issues related to skin color.

Contribution

It introduces a novel bias auditing framework that preserves temporal consistency and isolates visual attribute effects in HAR models, advancing fairness analysis methods.

Findings

HAR models can exhibit statistically significant biases related to skin color.

Controlled interventions reveal systematic errors across different groups.

The framework supports development of more transparent and accountable HAR systems.

Abstract

Human Action Recognition (HAR) models are increasingly deployed in high-stakes environments, yet their fairness across different human appearances has not been analyzed. We introduce a framework for auditing bias in HAR models using synthetic video data, generated with full control over visual identity attributes such as skin color. Unlike prior work that focuses on static images or pose estimation, our approach preserves temporal consistency, allowing us to isolate and test how changes to a single attribute affect model predictions. Through controlled interventions using the BEDLAM simulation platform, we show whether some popular HAR models exhibit statistically significant biases on the skin color even when the motion remains identical. Our results highlight how models may encode unwanted visual associations, and we provide evidence of systematic errors across groups. This work contributes a framework for auditing HAR models and supports the development of more transparent, accountable systems in light of upcoming regulatory standards.