Advancing Object Goal Navigation Through LLM-enhanced Object Affinities Transfer

TL;DR

This paper introduces LOAT, a framework that combines large language models with learned object relationships to improve object-goal navigation in robots, especially in unseen environments, showing significant success in simulations and real-world tests.

Contribution

The paper presents a novel dual-module approach that fuses LLM-derived knowledge with learned object affinities to enhance navigation generalization.

Findings

Improved navigation success rates in AI2-THOR and Habitat simulations.

Demonstrated zero-shot transfer capabilities in real-world environments.

Significant efficiency gains over existing methods.

Abstract

Object-goal navigation requires mobile robots to efficiently locate targets with visual and spatial information, yet existing methods struggle with generalization in unseen environments. Heuristic approaches with naive metrics fail in complex layouts, while graph-based and learning-based methods suffer from environmental biases and limited generalization. Although Large Language Models (LLMs) as planners or agents offer a rich knowledge base, they are cost-inefficient and lack targeted historical experience. To address these challenges, we propose the LLM-enhanced Object Affinities Transfer (LOAT) framework, integrating LLM-derived semantics with learning-based approaches to leverage experiential object affinities for better generalization in unseen settings. LOAT employs a dual-module strategy: one module accesses LLMs' vast knowledge, and the other applies learned object semantic relationships, dynamically fusing these sources based on context. Evaluations in AI2-THOR and Habitat simulators show significant improvements in navigation success and efficiency, and real-world deployment demonstrates the zero-shot ability of LOAT to enhance object-goal navigation systems.