Person Search in Videos with One Portrait Through Visual and Temporal Links

TL;DR

This paper introduces a novel person search framework in videos using a single portrait, leveraging visual and temporal links, and a new propagation scheme, significantly improving accuracy over existing methods.

Contribution

The paper proposes a new person search method that propagates identity information through visual and temporal links, and introduces a large-scale benchmark dataset for evaluation.

Findings

Achieved mAP of 62.27%, outperforming previous methods.

Developed a novel Progressive Propagation via Competitive Consensus scheme.

Constructed a large-scale dataset with 127K tracklets from 192 movies.

Abstract

In real-world applications, e.g. law enforcement and video retrieval, one often needs to search a certain person in long videos with just one portrait. This is much more challenging than the conventional settings for person re-identification, as the search may need to be carried out in the environments different from where the portrait was taken. In this paper, we aim to tackle this challenge and propose a novel framework, which takes into account the identity invariance along a tracklet, thus allowing person identities to be propagated via both the visual and the temporal links. We also develop a novel scheme called Progressive Propagation via Competitive Consensus, which significantly improves the reliability of the propagation process. To promote the study of person search, we construct a large-scale benchmark, which contains 127K manually annotated tracklets from 192 movies. Experiments show that our approach remarkably outperforms mainstream person re-id methods, raising the mAP from 42.16% to 62.27%.