VL-TGS: Trajectory Generation and Selection using Vision Language Models in Mapless Outdoor Environments

TL;DR

This paper introduces VL-TGS, a novel approach combining generative models and vision language models to enable robots to navigate complex outdoor environments in a human-like and safe manner without relying on maps.

Contribution

The paper proposes a multi-modal trajectory generation and selection framework that leverages CVAE and VLMs for mapless outdoor navigation, addressing environment-specific constraints and human-like path generation.

Findings

Achieved 20.81% improvement in traversability constraint satisfaction.

Achieved 28.51% improvement in human-like navigation.

Validated effectiveness across various outdoor scenes with wheeled robots.

Abstract

We present a multi-modal trajectory generation and selection algorithm for real-world mapless outdoor navigation in human-centered environments. Such environments contain rich features like crosswalks, grass, and curbs, which are easily interpretable by humans, but not by mobile robots. We aim to compute suitable trajectories that (1) satisfy the environment-specific traversability constraints and (2) generate human-like paths while navigating on crosswalks, sidewalks, etc. Our formulation uses a Conditional Variational Autoencoder (CVAE) generative model enhanced with traversability constraints to generate multiple candidate trajectories for global navigation. We develop a visual prompting approach and leverage the Visual Language Model's (VLM) zero-shot ability of semantic understanding and logical reasoning to choose the best trajectory given the contextual information about the task. We evaluate our method in various outdoor scenes with wheeled robots and compare the performance with other global navigation algorithms. In practice, we observe an average improvement of 20.81% in satisfying traversability constraints and 28.51% in terms of human-like navigation in four different outdoor navigation scenarios.