Embodied Navigation at the Art Gallery

TL;DR

This paper introduces ArtGallery3D, a new complex 3D environment of an art museum for embodied navigation, providing a challenging benchmark that reveals limitations of current methods and aims to foster future research.

Contribution

The paper presents a novel, richly detailed art museum environment for navigation benchmarks, with annotated points of interest and complex trajectories, expanding beyond existing indoor datasets.

Findings

Existing navigation methods struggle in the new environment.

Trajectories are more complex and longer than in previous datasets.

The environment highlights the need for improved navigation algorithms.

Abstract

Embodied agents, trained to explore and navigate indoor photorealistic environments, have achieved impressive results on standard datasets and benchmarks. So far, experiments and evaluations have involved domestic and working scenes like offices, flats, and houses. In this paper, we build and release a new 3D space with unique characteristics: the one of a complete art museum. We name this environment ArtGallery3D (AG3D). Compared with existing 3D scenes, the collected space is ampler, richer in visual features, and provides very sparse occupancy information. This feature is challenging for occupancy-based agents which are usually trained in crowded domestic environments with plenty of occupancy information. Additionally, we annotate the coordinates of the main points of interest inside the museum, such as paintings, statues, and other items. Thanks to this manual process, we deliver a new benchmark for PointGoal navigation inside this new space. Trajectories in this dataset are far more complex and lengthy than existing ground-truth paths for navigation in Gibson and Matterport3D. We carry on extensive experimental evaluation using our new space for evaluation and prove that existing methods hardly adapt to this scenario. As such, we believe that the availability of this 3D model will foster future research and help improve existing solutions.