Ground then Navigate: Language-guided Navigation in Dynamic Scenes

TL;DR

This paper introduces a novel approach for vision-and-language navigation in outdoor autonomous driving, grounding navigable regions via segmentation masks without relying on discretized maps, enhancing interpretability and maneuverability.

Contribution

It presents a new method that explicitly grounds navigable regions in visual scenes, moving beyond node selection to continuous action spaces, and introduces the CARLA-NAV dataset for training and validation.

Findings

Effective segmentation-based navigation in outdoor scenes

Improved interpretability through visual feedback

Validated with extensive empirical results

Abstract

We investigate the Vision-and-Language Navigation (VLN) problem in the context of autonomous driving in outdoor settings. We solve the problem by explicitly grounding the navigable regions corresponding to the textual command. At each timestamp, the model predicts a segmentation mask corresponding to the intermediate or the final navigable region. Our work contrasts with existing efforts in VLN, which pose this task as a node selection problem, given a discrete connected graph corresponding to the environment. We do not assume the availability of such a discretised map. Our work moves towards continuity in action space, provides interpretability through visual feedback and allows VLN on commands requiring finer manoeuvres like "park between the two cars". Furthermore, we propose a novel meta-dataset CARLA-NAV to allow efficient training and validation. The dataset comprises pre-recorded training sequences and a live environment for validation and testing. We provide extensive qualitative and quantitive empirical results to validate the efficacy of the proposed approach.