Learning Direct Control Policies with Flow Matching for Autonomous Driving

TL;DR

This paper introduces a flow-matching planner for autonomous driving that generates control trajectories directly from scene representations, enabling real-time re-planning and robust performance in unseen environments.

Contribution

The method uniquely combines flow-matching with BEV scene encoding to produce low-latency, generalizable control policies for autonomous driving.

Findings

Model generalizes well to unseen environments

Maintains stable control in out-of-distribution scenarios

Enables real-time closed-loop re-planning

Abstract

We present a flow-matching planner for autonomous driving that directly outputs actionable control trajectories defined by acceleration and curvature profiles. The model is conditioned on a bird's-eye-view (BEV) raster of the surrounding scene and generates control sequences in a small number of Ordinary Differential Equations (ODE) integration steps, enabling low-latency inference suitable for real-time closed-loop re-planning. We train exclusively on urban scenarios (real urban city streets, intersections and roundabouts of the city of Parma, Italy) collected from a 2D traffic simulator with reactive agents, and evaluate in closed-loop on both in-distribution and markedly out-of-distribution environments, including multi-lane highways and unseen urban scenarios. Our results show that the model generalizes reliably to these unseen conditions, maintaining stable closed-loop control and successfully completing scenarios that differ substantially from the training distribution. We attribute this to the BEV representation, which provides a geometry-centric view of the scene that is inherently less sensitive to distributional shifts, and to the flow-matching formulation, which learns a smooth vector field that degrades gracefully under distribution shift. We provide video demonstrations of closed-loop behavior at https://marcelloceresini.github.io/DirectControlFlowMatching.