Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehicles

TL;DR

This paper introduces the Risk Ranked Recall ($R^3$) metric, a new evaluation method for object detection in autonomous vehicles that considers collision risk to better assess safety performance.

Contribution

The paper proposes the $R^3$ metric, which categorizes objects by collision risk and measures recall within each risk rank, providing a more safety-oriented evaluation.

Findings

$R^3$ offers a risk-aware assessment of object detection systems.

It improves understanding of detection performance in safety-critical scenarios.

The metric aligns evaluation with collision avoidance priorities.

Abstract

Commonly used metrics for evaluation of object detection systems (precision, recall, mAP) do not give complete information about their suitability of use in safety critical tasks, like obstacle detection for collision avoidance in Autonomous Vehicles (AV). This work introduces the Risk Ranked Recall ($R^3$) metrics for object detection systems. The $R^3$ metrics categorize objects within three ranks. Ranks are assigned based on an objective cyber-physical model for the risk of collision. Recall is measured for each rank.