GET: Goal-directed Exploration and Targeting for Large-Scale Unknown Environments

TL;DR

GET is a novel framework that combines language-based reasoning with experience-guided exploration to improve object search efficiency in large, dynamic environments, addressing the grounding and memory challenges of LLMs in robotics.

Contribution

The paper introduces GET, integrating a reasoning module and probabilistic memory to enhance LLM-based embodied exploration in large-scale environments.

Findings

GET outperforms heuristic and LLM-only baselines in real-world tests.

Structured LLM integration improves decision-making in complex environments.

GET demonstrates robustness across multiple LLMs and task settings.

Abstract

Object search in large-scale, unstructured environments remains a fundamental challenge in robotics, particularly in dynamic or expansive settings such as outdoor autonomous exploration. This task requires robust spatial reasoning and the ability to leverage prior experiences. While Large Language Models (LLMs) offer strong semantic capabilities, their application in embodied contexts is limited by a grounding gap in spatial reasoning and insufficient mechanisms for memory integration and decision consistency.To address these challenges, we propose GET (Goal-directed Exploration and Targeting), a framework that enhances object search by combining LLM-based reasoning with experience-guided exploration. At its core is DoUT (Diagram of Unified Thought), a reasoning module that facilitates real-time decision-making through a role-based feedback loop, integrating task-specific criteria and external memory. For repeated tasks, GET maintains a probabilistic task map based on a Gaussian Mixture Model, allowing for continual updates to object-location priors as environments evolve.Experiments conducted in real-world, large-scale environments demonstrate that GET improves search efficiency and robustness across multiple LLMs and task settings, significantly outperforming heuristic and LLM-only baselines. These results suggest that structured LLM integration provides a scalable and generalizable approach to embodied decision-making in complex environments.