Towards Navigation by Reasoning over Spatial Configurations

TL;DR

This paper introduces a neural navigation agent that leverages spatial configurations and reasoning to better understand natural language instructions and improve navigation performance in both seen and unseen environments.

Contribution

It presents a novel approach that explicitly models spatial semantics and execution order, enhancing language grounding and spatial reasoning in navigation tasks.

Findings

Improved performance on seen environments

Competitive results on unseen environments

Explicit spatial modeling enhances reasoning

Abstract

We deal with the navigation problem where the agent follows natural language instructions while observing the environment. Focusing on language understanding, we show the importance of spatial semantics in grounding navigation instructions into visual perceptions. We propose a neural agent that uses the elements of spatial configurations and investigate their influence on the navigation agent's reasoning ability. Moreover, we model the sequential execution order and align visual objects with spatial configurations in the instruction. Our neural agent improves strong baselines on the seen environments and shows competitive performance on the unseen environments. Additionally, the experimental results demonstrate that explicit modeling of spatial semantic elements in the instructions can improve the grounding and spatial reasoning of the model.