A Quality Index Metric and Method for Online Self-Assessment of Autonomous Vehicles Sensory Perception

TL;DR

This paper introduces a novel detection quality index (DQI) and a superpixel-based attention network (SPA-NET) to evaluate and predict camera-based object detection performance in autonomous vehicles, providing frame-by-frame feedback.

Contribution

The paper presents a new DQI metric combined with SPA-NET for real-time, frame-level assessment of object detection quality in autonomous driving scenarios.

Findings

DQI accurately assesses detection quality in autonomous driving datasets.

SPA-NET outperforms other image-based quality regression models.

The approach provides valuable self-evaluation for camera perception systems.

Abstract

Reliable object detection using cameras plays a crucial role in enabling autonomous vehicles to perceive their surroundings. However, existing camera-based object detection approaches for autonomous driving lack the ability to provide comprehensive feedback on detection performance for individual frames. To address this limitation, we propose a novel evaluation metric, named as the detection quality index (DQI), which assesses the performance of camera-based object detection algorithms and provides frame-by-frame feedback on detection quality. The DQI is generated by combining the intensity of the fine-grained saliency map with the output results of the object detection algorithm. Additionally, we have developed a superpixel-based attention network (SPA-NET) that utilizes raw image pixels and superpixels as input to predict the proposed DQI evaluation metric. To validate our approach, we conducted experiments on three open-source datasets. The results demonstrate that the proposed evaluation metric accurately assesses the detection quality of camera-based systems in autonomous driving environments. Furthermore, the proposed SPA-NET outperforms other popular image-based quality regression models. This highlights the effectiveness of the DQI in evaluating a camera's ability to perceive visual scenes. Overall, our work introduces a valuable self-evaluation tool for camera-based object detection in autonomous vehicles.