ZeST: an LLM-based Zero-Shot Traversability Navigation for Unknown Environments

TL;DR

ZeST utilizes large language models' visual reasoning to generate real-time traversability maps for autonomous navigation, eliminating the need for risky real-world data collection and enhancing safety and scalability.

Contribution

This paper introduces ZeST, a novel LLM-based zero-shot method for traversability prediction that avoids hazardous environment data collection and improves navigation safety.

Findings

Safer navigation in controlled and outdoor environments

Achieves comparable or better success rates than state-of-the-art methods

Reduces risks and costs associated with traditional data collection

Abstract

The advancement of robotics and autonomous navigation systems hinges on the ability to accurately predict terrain traversability. Traditional methods for generating datasets to train these prediction models often involve putting robots into potentially hazardous environments, posing risks to equipment and safety. To solve this problem, we present ZeST, a novel approach leveraging visual reasoning capabilities of Large Language Models (LLMs) to create a traversability map in real-time without exposing robots to danger. Our approach not only performs zero-shot traversability and mitigates the risks associated with real-world data collection but also accelerates the development of advanced navigation systems, offering a cost-effective and scalable solution. To support our findings, we present navigation results, in both controlled indoor and unstructured outdoor environments. As shown in the experiments, our method provides safer navigation when compared to other state-of-the-art methods, constantly reaching the final goal.