Checklist to Define the Identification of TP, FP, and FN Object Detections in Automated Driving

TL;DR

This paper presents a comprehensive checklist for defining true positives, false positives, and false negatives in object detection for automated driving, aiming to improve test reliability and comparability.

Contribution

It offers a detailed checklist covering functional aspects and implementation details to standardize detection evaluation in automated driving systems.

Findings

Provides a structured checklist for detection evaluation

Helps reduce ambiguity in object perception tests

Enhances comparability of detection metrics

Abstract

The object perception of automated driving systems must pass quality and robustness tests before a safe deployment. Such tests typically identify true positive (TP), false-positive (FP), and false-negative (FN) detections and aggregate them to metrics. Since the literature seems to be lacking a comprehensive way to define the identification of TPs/FPs/FNs, this paper provides a checklist of relevant functional aspects and implementation details. Besides labeling policies of the test set, we cover areas of vision, occlusion handling, safety-relevant areas, matching criteria, temporal and probabilistic issues, and further aspects. Even though the checklist cannot be fully formalized, it can help practitioners minimize the ambiguity of their tests, which, in turn, makes statements on object perception more reliable and comparable.