NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Data

TL;DR

NymeriaPlus enhances the Nymeria dataset by adding detailed annotations, multiple modalities, and improved motion data, aiming to support advanced egocentric and multimodal AI research.

Contribution

This paper introduces NymeriaPlus, a significantly enriched version of the Nymeria dataset with new annotations, modalities, and improved motion data, filling gaps in existing egocentric datasets.

Findings

Enhanced human motion data in MHR and SMPL formats.

Added dense 3D/2D object annotations and reconstructions.

Inclusion of multimodal data like audio and basemaps.

Abstract

The Nymeria Dataset, released in 2024, is a large-scale collection of in-the-wild human activities captured with multiple egocentric wearable devices that are spatially localized and temporally synchronized. It provides body-motion ground truth recorded with a motion-capture suit, device trajectories, semi-dense 3D point clouds, and in-context narrations. In this paper, we upgrade Nymeria and introduce NymeriaPlus. NymeriaPlus features: (1) improved human motion in Momentum Human Rig (MHR) and SMPL formats; (2) dense 3D and 2D bounding box annotations for indoor objects and structural elements; (3) instance-level 3D object reconstructions; and (4) additional modalities e.g., basemap recordings, audio, and wristband videos. By consolidating these complementary modalities and annotations into a single, coherent benchmark, NymeriaPlus strengthens Nymeria into a more powerful in-the-wild egocentric dataset. We expect NymeriaPlus to bridge a key gap in existing egocentric resources and to support a broader range of research, including unique explorations of multimodal learning for embodied AI.