ENIGMA-51: Towards a Fine-Grained Understanding of Human-Object Interactions in Industrial Scenarios

TL;DR

ENIGMA-51 is a comprehensive egocentric dataset capturing human-object interactions in industrial repair tasks, enabling advanced analysis and benchmarking of behavior understanding in industrial environments.

Contribution

The paper introduces ENIGMA-51, a new dataset with detailed annotations for studying human-object interactions in industrial scenarios, along with baseline benchmarks for multiple tasks.

Findings

Dataset is challenging for current models.

Benchmarks established for interaction detection and anticipation.

Baseline results highlight the need for improved methods.

Abstract

ENIGMA-51 is a new egocentric dataset acquired in an industrial scenario by 19 subjects who followed instructions to complete the repair of electrical boards using industrial tools (e.g., electric screwdriver) and equipments (e.g., oscilloscope). The 51 egocentric video sequences are densely annotated with a rich set of labels that enable the systematic study of human behavior in the industrial domain. We provide benchmarks on four tasks related to human behavior: 1) untrimmed temporal detection of human-object interactions, 2) egocentric human-object interaction detection, 3) short-term object interaction anticipation and 4) natural language understanding of intents and entities. Baseline results show that the ENIGMA-51 dataset poses a challenging benchmark to study human behavior in industrial scenarios. We publicly release the dataset at https://iplab.dmi.unict.it/ENIGMA-51.