EgoObjects: A Large-Scale Egocentric Dataset for Fine-Grained Object Understanding

TL;DR

EgoObjects is a comprehensive large-scale egocentric dataset with fine-grained object annotations, designed to advance research in egocentric object understanding through diverse data and benchmark tasks.

Contribution

The paper introduces EgoObjects, a new large-scale egocentric dataset with detailed annotations and multiple benchmark tasks, including novel continual learning challenges.

Findings

Over 650K object annotations across 368 categories

Includes instance-level identifiers for over 14K objects

Supports multiple benchmark tasks for egocentric object understanding

Abstract

Object understanding in egocentric visual data is arguably a fundamental research topic in egocentric vision. However, existing object datasets are either non-egocentric or have limitations in object categories, visual content, and annotation granularities. In this work, we introduce EgoObjects, a large-scale egocentric dataset for fine-grained object understanding. Its Pilot version contains over 9K videos collected by 250 participants from 50+ countries using 4 wearable devices, and over 650K object annotations from 368 object categories. Unlike prior datasets containing only object category labels, EgoObjects also annotates each object with an instance-level identifier, and includes over 14K unique object instances. EgoObjects was designed to capture the same object under diverse background complexities, surrounding objects, distance, lighting and camera motion. In parallel to the data collection, we conducted data annotation by developing a multi-stage federated annotation process to accommodate the growing nature of the dataset. To bootstrap the research on EgoObjects, we present a suite of 4 benchmark tasks around the egocentric object understanding, including a novel instance level- and the classical category level object detection. Moreover, we also introduce 2 novel continual learning object detection tasks. The dataset and API are available at https://github.com/facebookresearch/EgoObjects.