Keypoint Promptable Re-Identification

TL;DR

This paper introduces Keypoint Promptable ReID (KPR), a new approach for occluded person re-identification that uses semantic keypoints to resolve multi-person ambiguity, supported by a new dataset and annotations, outperforming previous methods.

Contribution

The paper proposes a novel prompt-based ReID formulation using keypoints and introduces a new dataset with keypoint annotations for occluded scenarios.

Findings

KPR outperforms previous state-of-the-art methods on occluded ReID tasks.

The new dataset enables effective training and evaluation of keypoint-based ReID methods.

Using keypoints reduces multi-person ambiguity in occluded ReID scenarios.

Abstract

Occluded Person Re-Identification (ReID) is a metric learning task that involves matching occluded individuals based on their appearance. While many studies have tackled occlusions caused by objects, multi-person occlusions remain less explored. In this work, we identify and address a critical challenge overlooked by previous occluded ReID methods: the Multi-Person Ambiguity (MPA) arising when multiple individuals are visible in the same bounding box, making it impossible to determine the intended ReID target among the candidates. Inspired by recent work on prompting in vision, we introduce Keypoint Promptable ReID (KPR), a novel formulation of the ReID problem that explicitly complements the input bounding box with a set of semantic keypoints indicating the intended target. Since promptable re-identification is an unexplored paradigm, existing ReID datasets lack the pixel-level annotations necessary for prompting. To bridge this gap and foster further research on this topic, we introduce Occluded-PoseTrack ReID, a novel ReID dataset with keypoints labels, that features strong inter-person occlusions. Furthermore, we release custom keypoint labels for four popular ReID benchmarks. Experiments on person retrieval, but also on pose tracking, demonstrate that our method systematically surpasses previous state-of-the-art approaches on various occluded scenarios. Our code, dataset and annotations are available at https://github.com/VlSomers/keypoint_promptable_reidentification.