ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarios

TL;DR

ENIGMA-360 introduces a novel synchronized ego-exo dataset from industrial environments, enabling advanced human behavior analysis and benchmarking for real-world applications.

Contribution

The paper presents ENIGMA-360, a new dataset capturing synchronized egocentric and exocentric videos in industrial scenarios, with annotations for behavior understanding tasks.

Findings

Baseline experiments reveal limitations of current models in industrial settings.

The dataset exposes the need for more robust ego-exo understanding models.

Public release of the dataset facilitates future research in this domain.

Abstract

Understanding human behavior from complementary egocentric (ego) and exocentric (exo) points of view enables the development of systems that can support workers in industrial environments and enhance their safety. However, progress in this area is hindered by the lack of datasets capturing both views in realistic industrial scenarios. To address this gap, we propose ENIGMA-360, a new ego-exo dataset acquired in a real industrial scenario. The dataset is composed of 180 egocentric and 180 exocentric procedural videos temporally synchronized offering complementary information of the same scene. The 360 videos have been labeled with temporal and spatial annotations, enabling the study of different aspects of human behavior in industrial domain. We provide baseline experiments for 3 foundational tasks for human behavior understanding: 1) Temporal Action Segmentation, 2) Keystep Recognition and 3) Egocentric Human-Object Interaction Detection, showing the limits of state-of-the-art approaches on this challenging scenario. These results highlight the need for new models capable of robust ego-exo understanding in real-world environments. We publicly release the dataset and its annotations at https://fpv-iplab.github.io/ENIGMA-360/.