Diffusion-FS: Multimodal Free-Space Prediction via Diffusion for Autonomous Driving

TL;DR

This paper introduces Diffusion-FS, a novel monocular camera-based method for predicting navigable free-space corridors in autonomous driving using a diffusion process over contour points, with self-supervised data generation.

Contribution

It proposes a self-supervised approach for corridor data generation and a diffusion-based architecture, ContourDiff, for structured free-space prediction from monocular images.

Findings

Effective multimodal corridor prediction demonstrated on nuScenes and CARLA datasets.

Outperforms existing methods in accuracy and interpretability.

Enables monocular perception-based autonomous navigation.

Abstract

Drivable Free-space prediction is a fundamental and crucial problem in autonomous driving. Recent works have addressed the problem by representing the entire non-obstacle road regions as the free-space. In contrast our aim is to estimate the driving corridors that are a navigable subset of the entire road region. Unfortunately, existing corridor estimation methods directly assume a BEV-centric representation, which is hard to obtain. In contrast, we frame drivable free-space corridor prediction as a pure image perception task, using only monocular camera input. However such a formulation poses several challenges as one doesn't have the corresponding data for such free-space corridor segments in the image. Consequently, we develop a novel self-supervised approach for free-space sample generation by leveraging future ego trajectories and front-view camera images, making the process of visual corridor estimation dependent on the ego trajectory. We then employ a diffusion process to model the distribution of such segments in the image. However, the existing binary mask-based representation for a segment poses many limitations. Therefore, we introduce ContourDiff, a specialized diffusion-based architecture that denoises over contour points rather than relying on binary mask representations, enabling structured and interpretable free-space predictions. We evaluate our approach qualitatively and quantitatively on both nuScenes and CARLA, demonstrating its effectiveness in accurately predicting safe multimodal navigable corridors in the image.