Learning Feature Recovery Transformer for Occluded Person Re-identification

TL;DR

This paper introduces the Feature Recovery Transformer (FRT), a novel method for occluded person re-identification that effectively recovers complete features and improves matching accuracy by focusing on visible regions and leveraging neighbor information.

Contribution

The paper proposes a new FRT approach combining visibility graph matching and feature recovery transformer to address occlusion challenges in person Re-ID, outperforming existing methods.

Findings

FRT achieves at least 6.2% higher Rank-1 accuracy on Occluded-Duke.

FRT improves mAP scores by 7.2% on occluded datasets.

Extensive experiments validate FRT's effectiveness across various datasets.

Abstract

One major issue that challenges person re-identification (Re-ID) is the ubiquitous occlusion over the captured persons. There are two main challenges for the occluded person Re-ID problem, i.e., the interference of noise during feature matching and the loss of pedestrian information brought by the occlusions. In this paper, we propose a new approach called Feature Recovery Transformer (FRT) to address the two challenges simultaneously, which mainly consists of visibility graph matching and feature recovery transformer. To reduce the interference of the noise during feature matching, we mainly focus on visible regions that appear in both images and develop a visibility graph to calculate the similarity. In terms of the second challenge, based on the developed graph similarity, for each query image, we propose a recovery transformer that exploits the feature sets of its $k$-nearest neighbors in the gallery to recover the complete features. Extensive experiments across different person Re-ID datasets, including occluded, partial and holistic datasets, demonstrate the effectiveness of FRT. Specifically, FRT significantly outperforms state-of-the-art results by at least 6.2\% Rank-1 accuracy and 7.2\% mAP scores on the challenging Occluded-Duke dataset. The code is available at https://github.com/xbq1994/Feature-Recovery-Transformer.