Learning to Reason and Navigate: Parameter Efficient Action Planning with Large Language Models

TL;DR

This paper introduces PEAP-LLM, a parameter-efficient, two-module approach using large language models for goal-oriented navigation in complex indoor environments, significantly improving performance on the REVERIE task.

Contribution

It presents a novel two-stage fine-tuning method for LLMs and a modular action planning framework for embodied navigation without pre-exploration.

Findings

PEAP-LLM outperforms previous state-of-the-art on REVERIE.

The two-stage fine-tuning improves instruction quality and environmental adaptability.

Modular LLM-based planning enhances navigation success in complex scenarios.

Abstract

The remote embodied referring expression (REVERIE) task requires an agent to navigate through complex indoor environments and localize a remote object specified by high-level instructions, such as "bring me a spoon", without pre-exploration. Hence, an efficient navigation plan is essential for the final success. This paper proposes a novel parameter-efficient action planner using large language models (PEAP-LLM) to generate a single-step instruction at each location. The proposed model consists of two modules, LLM goal planner (LGP) and LoRA action planner (LAP). Initially, LGP extracts the goal-oriented plan from REVERIE instructions, including the target object and room. Then, LAP generates a single-step instruction with the goal-oriented plan, high-level instruction, and current visual observation as input. PEAP-LLM enables the embodied agent to interact with LAP as the path planner on the fly. A simple direct application of LLMs hardly achieves good performance. Also, existing hard-prompt-based methods are error-prone in complicated scenarios and need human intervention. To address these issues and prevent the LLM from generating hallucinations and biased information, we propose a novel two-stage method for fine-tuning the LLM, consisting of supervised fine-tuning (STF) and direct preference optimization (DPO). SFT improves the quality of generated instructions, while DPO utilizes environmental feedback. Experimental results show the superiority of our proposed model on REVERIE compared to the previous state-of-the-art.