Benchmarking person re-identification datasets and approaches for practical real-world implementations

TL;DR

This paper evaluates the robustness of person re-identification models across different real-world environments, highlighting domain shift challenges and proposing a benchmarking methodology for practical deployment.

Contribution

It introduces a comprehensive evaluation methodology for Re-ID approaches and datasets, focusing on their effectiveness in real-world, unsupervised scenarios.

Findings

Benchmarking of four Re-ID approaches on three datasets

Insights into domain shift impacts on Re-ID performance

Guidelines for designing more robust Re-ID systems

Abstract

Recently, Person Re-Identification (Re-ID) has received a lot of attention. Large datasets containing labeled images of various individuals have been released, allowing researchers to develop and test many successful approaches. However, when such Re-ID models are deployed in new cities or environments, the task of searching for people within a network of security cameras is likely to face an important domain shift, thus resulting in decreased performance. Indeed, while most public datasets were collected in a limited geographic area, images from a new city present different features (e.g., people's ethnicity and clothing style, weather, architecture, etc.). In addition, the whole frames of the video streams must be converted into cropped images of people using pedestrian detection models, which behave differently from the human annotators who created the dataset used for training. To better understand the extent of this issue, this paper introduces a complete methodology to evaluate Re-ID approaches and training datasets with respect to their suitability for unsupervised deployment for live operations. This method is used to benchmark four Re-ID approaches on three datasets, providing insight and guidelines that can help to design better Re-ID pipelines in the future.