Diffusion as Reasoning: Enhancing Object Navigation via Diffusion Model Conditioned on LLM-based Object-Room Knowledge

TL;DR

This paper introduces a novel diffusion-based approach combined with LLM-derived knowledge to improve object navigation by generating semantic maps of unexplored areas, leading to better long-term goal reasoning.

Contribution

It proposes a diffusion model conditioned on explored maps and LLM-based room guidance to enhance object navigation in unseen environments.

Findings

Significant improvement in navigation success rates on Gibson and MP3D datasets.

Effective long-term goal reasoning through diffusion-generated semantic maps.

Leveraging LLMs for room-aware object distribution guidance.

Abstract

The Object Navigation (ObjectNav) task aims to guide an agent to locate target objects in unseen environments using partial observations. Prior approaches have employed location prediction paradigms to achieve long-term goal reasoning, yet these methods often struggle to effectively integrate contextual relation reasoning. Alternatively, map completion-based paradigms predict long-term goals by generating semantic maps of unexplored areas. However, existing methods in this category fail to fully leverage known environmental information, resulting in suboptimal map quality that requires further improvement. In this work, we propose a novel approach to enhancing the ObjectNav task, by training a diffusion model to learn the statistical distribution patterns of objects in semantic maps, and using the map of the explored regions during navigation as the condition to generate the map of the unknown regions, thereby realizing the long-term goal reasoning of the target object, i.e., diffusion as reasoning (DAR). Meanwhile, we propose the Room Guidance method, which leverages commonsense knowledge derived from large language models (LLMs) to guide the diffusion model in generating room-aware object distributions. Based on the generated map in the unknown region, the agent sets the predicted location of the target as the goal and moves towards it. Experiments on Gibson and MP3D show the effectiveness of our method.