CogExplore: Contextual Exploration with Language-Encoded Environment Representations

TL;DR

This paper introduces CogExplore, a language-based robotic exploration method that uses large language models to improve efficiency and robustness in search-and-rescue scenarios by reasoning over semantic and temporal cues.

Contribution

It presents a novel approach integrating large language models into exploration strategies, enabling semantic reasoning and environment-dependent heuristics for better exploration performance.

Findings

Reduces exploration path distance significantly

Achieves 100% success rate across diverse environments

Robust to noisy vision detections

Abstract

Integrating language models into robotic exploration frameworks improves performance in unmapped environments by providing the ability to reason over semantic groundings, contextual cues, and temporal states. The proposed method employs large language models (GPT-3.5 and Claude Haiku) to reason over these cues and express that reasoning in terms of natural language, which can be used to inform future states. We are motivated by the context of search-and-rescue applications where efficient exploration is critical. We find that by leveraging natural language, semantics, and tracking temporal states, the proposed method greatly reduces exploration path distance and further exposes the need for environment-dependent heuristics. Moreover, the method is highly robust to a variety of environments and noisy vision detections, as shown with a 100% success rate in a series of comprehensive experiments across three different environments conducted in a custom simulation pipeline operating in Unreal Engine.