Revisiting the Evaluation of Deep Neural Networks for Pedestrian Detection

TL;DR

This paper introduces new error categories and metrics for evaluating pedestrian detection DNNs, enabling more detailed and safety-focused performance comparisons, and achieves state-of-the-art results on CityPersons.

Contribution

It proposes a novel error categorization and metrics for pedestrian detection evaluation, improving robustness and safety relevance of model comparisons.

Findings

New error categories for pedestrian detection evaluation

Proposed metrics enable fine-grained performance analysis

Achieved state-of-the-art results on CityPersons dataset

Abstract

Reliable pedestrian detection represents a crucial step towards automated driving systems. However, the current performance benchmarks exhibit weaknesses. The currently applied metrics for various subsets of a validation dataset prohibit a realistic performance evaluation of a DNN for pedestrian detection. As image segmentation supplies fine-grained information about a street scene, it can serve as a starting point to automatically distinguish between different types of errors during the evaluation of a pedestrian detector. In this work, eight different error categories for pedestrian detection are proposed and new metrics are proposed for performance comparison along these error categories. We use the new metrics to compare various backbones for a simplified version of the APD, and show a more fine-grained and robust way to compare models with each other especially in terms of safety-critical performance. We achieve SOTA on CityPersons-reasonable (without extra training data) by using a rather simple architecture.