CoFL: Continuous Flow Fields for Language-Conditioned Navigation

TL;DR

CoFL introduces an end-to-end navigation policy that learns continuous flow fields from BEV observations and language instructions, enabling real-time, zero-shot deployment in unseen scenes with improved safety and precision.

Contribution

It reformulates language-conditioned navigation as workspace-conditioned flow field learning, enabling dense spatial control supervision and robust real-time navigation.

Findings

Outperforms modular VLM-based planners in unseen scenes

Enables real-time, zero-shot deployment in real-world environments

Builds a large dataset of 500k BEV instruction-flow pairs for training

Abstract

Existing language-conditioned navigation systems typically rely on modular pipelines or trajectory generators, but the latter use each scene--instruction annotation mainly to supervise one start-conditioned rollout. To address these limitations, we present CoFL, an end-to-end policy that maps a bird's-eye view (BEV) observation and a language instruction to a continuous flow field for navigation. CoFL reformulates navigation as workspace-conditioned field learning rather than start-conditioned trajectory prediction: it learns local motion vectors at arbitrary BEV locations, turning each scene--instruction annotation into dense spatial control supervision. Trajectories are generated from any start by numerical integration of the predicted field, enabling simple real-time rollout and closed-loop recovery. To enable large-scale training and evaluation, we build a dataset of over 500k BEV image--instruction pairs, each procedurally annotated with a flow field and a trajectory derived from semantic maps built on Matterport3D and ScanNet. Evaluating on strictly unseen scenes, CoFL significantly outperforms modular Vision-Language Model (VLM)-based planners and trajectory generation policies in both navigation precision and safety, while maintaining real-time inference. Finally, we deploy CoFL zero-shot in real-world experiments with BEV observations across multiple layouts, maintaining feasible closed-loop control and a high success rate.