Risk Aware Safe Control with Multi-Modal Sensing for Dynamic Obstacle Avoidance

TL;DR

This paper introduces a risk-aware control framework for autonomous vehicles that combines multi-modal sensing data with probabilistic safety measures to enhance obstacle avoidance in dynamic environments.

Contribution

It presents a novel integration of multi-modal sensing, Wasserstein barycenter-based probabilistic estimation, and CVaR-based safety filtering for autonomous vehicle control.

Findings

Improved safety and robustness over baseline methods.

Average 12.7% success rate increase in obstacle avoidance scenarios.

Validated on a full-scale autonomous vehicle.

Abstract

Safe control in dynamic traffic environments remains a major challenge for autonomous vehicles (AVs), as ego vehicle and obstacle states are inherently affected by sensing noise and estimation uncertainty. However, existing studies have not sufficiently addressed how uncertain multi-modal sensing information can be systematically incorporated into tail-risk-aware safety-critical control. To address this gap, this paper proposes a risk-aware safe control framework that integrates probabilistic state estimation with a conditional value-at-risk (CVaR) control barrier function (CBF) safety filter. Obstacle detections from cameras, LiDAR, and vehicle-to-everything (V2X) communication are combined using a Wasserstein barycenter (WB) to obtain a probabilistic state estimate. A model predictive controller generates the nominal control, which is then filtered through a CVaR-CBF quadratic program to enforce risk-aware safety constraints. The approach is evaluated through numerical studies and further validated on a full-scale AV. Results demonstrate improved safety and robustness over a baseline MPC-CBF design, with an average improvement of 12.7\% in success rate across the evaluated scenarios.