Semantic Environment Atlas for Object-Goal Navigation

TL;DR

This paper presents the Semantic Environment Atlas (SEA), a novel semantic mapping approach that improves visual localization and object-goal navigation for embodied agents by using semantic graph maps.

Contribution

The paper introduces SEA, a new semantic graph mapping method that enhances navigation and localization by capturing place-object relationships from images.

Findings

SEA significantly outperforms existing localization methods.

Achieves 39.0% success rate in Habitat scenarios, 12.4% higher than state-of-the-art.

Maintains robustness under noisy conditions with low computational costs.

Abstract

In this paper, we introduce the Semantic Environment Atlas (SEA), a novel mapping approach designed to enhance visual navigation capabilities of embodied agents. The SEA utilizes semantic graph maps that intricately delineate the relationships between places and objects, thereby enriching the navigational context. These maps are constructed from image observations and capture visual landmarks as sparsely encoded nodes within the environment. The SEA integrates multiple semantic maps from various environments, retaining a memory of place-object relationships, which proves invaluable for tasks such as visual localization and navigation. We developed navigation frameworks that effectively leverage the SEA, and we evaluated these frameworks through visual localization and object-goal navigation tasks. Our SEA-based localization framework significantly outperforms existing methods, accurately identifying locations from single query images. Experimental results in Habitat scenarios show that our method not only achieves a success rate of 39.0%, an improvement of 12.4% over the current state-of-the-art, but also maintains robustness under noisy odometry and actuation conditions, all while keeping computational costs low.