Sem-NaVAE: Semantically-Guided Outdoor Mapless Navigation via Generative Trajectory Priors

TL;DR

Sem-NaVAE introduces a mapless outdoor navigation system that combines generative trajectory models with semantic language understanding to enable real-time, flexible, and natural language-guided navigation in outdoor environments.

Contribution

It is the first to integrate CVAEs with semantic segmentation for outdoor mapless navigation, allowing real-time trajectory generation and selection based on natural language commands.

Findings

Achieved superior outdoor navigation performance compared to existing methods.

Enabled real-time trajectory generation and selection using semantic cues.

Validated system effectiveness through real-world outdoor experiments.

Abstract

This work presents a mapless global navigation approach for outdoor applications. It combines the exploratory capacity of conditional variational autoencoders (CVAEs) to generate trajectories and the semantic segmentation capabilities of a lightweight visual language model (VLM) to select the trajectory to execute. Open-vocabulary segmentation is used to score and select the generated trajectories based on natural language, and a state-of-the-art local planner executes velocity commands. One of the key features of the proposed approach is its ability to generate a large variability of trajectories and to select them and navigate in real-time. The approach was validated through real-world outdoor navigation experiments, achieving superior performance compared to state-of-the-art methods. A video showing an experimental run of the system can be found in https://www.youtube.com/watch?v=i3R5ey5O2yk.