Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Prediction

TL;DR

This paper introduces Conformal Risk Tube Prediction, a novel framework for modeling spatiotemporal risk uncertainty in vision-based hazard detection for autonomous driving, improving safety and robustness.

Contribution

It presents a unified conformal prediction approach that captures risk uncertainty across space and time with coverage guarantees, along with a new dataset and metrics for evaluation.

Findings

Enhanced robustness in risk object identification

Reduced nuisance braking alerts in downstream tasks

Significant improvement over prior methods in uncertainty estimation

Abstract

We study object importance-based vision risk object identification (Vision-ROI), a key capability for hazard detection in intelligent driving systems. Existing approaches make deterministic decisions and ignore uncertainty, which could lead to safety-critical failures. Specifically, in ambiguous scenarios, fixed decision thresholds may cause premature or delayed risk detection and temporally unstable predictions, especially in complex scenes with multiple interacting risks. Despite these challenges, current methods lack a principled framework to model risk uncertainty jointly across space and time. We propose Conformal Risk Tube Prediction, a unified formulation that captures spatiotemporal risk uncertainty, provides coverage guarantees for true risks, and produces calibrated risk scores with uncertainty estimates. To conduct a systematic evaluation, we present a new dataset and metrics probing diverse scenario configurations with multi-risk coupling effects, which are not supported by existing datasets. We systematically analyze factors affecting uncertainty estimation, including scenario variations, per-risk category behavior, and perception error propagation. Our method delivers substantial improvements over prior approaches, enhancing vision-ROI robustness and downstream performance, such as reducing nuisance braking alerts. For more qualitative results, please visit our project webpage: https://hcis-lab.github.io/CRTP/