The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain

TL;DR

The MECCANO dataset provides a new egocentric video benchmark for understanding human-object interactions in industrial-like settings, enabling research on action recognition and object detection from a first-person perspective.

Contribution

We introduce MECCANO, the first egocentric dataset focused on industrial scenarios, with detailed annotations for multiple human-object interaction tasks.

Findings

Baseline results highlight the dataset's challenge for egocentric interaction tasks.

The dataset supports four key tasks: action recognition, active object detection, recognition, and interaction detection.

Public release of the dataset facilitates further research in egocentric industrial applications.

Abstract

Wearable cameras allow to collect images and videos of humans interacting with the world. While human-object interactions have been thoroughly investigated in third person vision, the problem has been understudied in egocentric settings and in industrial scenarios. To fill this gap, we introduce MECCANO, the first dataset of egocentric videos to study human-object interactions in industrial-like settings. MECCANO has been acquired by 20 participants who were asked to build a motorbike model, for which they had to interact with tiny objects and tools. The dataset has been explicitly labeled for the task of recognizing human-object interactions from an egocentric perspective. Specifically, each interaction has been labeled both temporally (with action segments) and spatially (with active object bounding boxes). With the proposed dataset, we investigate four different tasks including 1) action recognition, 2) active object detection, 3) active object recognition and 4) egocentric human-object interaction detection, which is a revisited version of the standard human-object interaction detection task. Baseline results show that the MECCANO dataset is a challenging benchmark to study egocentric human-object interactions in industrial-like scenarios. We publicy release the dataset at https://iplab.dmi.unict.it/MECCANO.