Beyond Domain Gap: Exploiting Subjectivity in Sketch-Based Person Retrieval

TL;DR

This paper introduces a large-scale, multi-style sketch re-identification dataset with multiple witness perspectives, and proposes novel modules to address subjectivity, advancing the robustness and accuracy of sketch-based person retrieval.

Contribution

It presents the first large-scale multi-witness sketch re-ID dataset and introduces fusion and alignment modules to handle subjectivity and style variations.

Findings

The dataset contains over 4,763 sketches and 32,668 photos.

Proposed modules improve performance on multi-style and cross-style benchmarks.

Achieved state-of-the-art results in sketch re-ID tasks.

Abstract

Person re-identification (re-ID) requires densely distributed cameras. In practice, the person of interest may not be captured by cameras and, therefore, needs to be retrieved using subjective information (e.g., sketches from witnesses). Previous research defines this case using the sketch as sketch re-identification (Sketch re-ID) and focuses on eliminating the domain gap. Actually, subjectivity is another significant challenge. We model and investigate it by posing a new dataset with multi-witness descriptions. It features two aspects. 1) Large-scale. It contains over 4,763 sketches and 32,668 photos, making it the largest Sketch re-ID dataset. 2) Multi-perspective and multi-style. Our dataset offers multiple sketches for each identity. Witnesses' subjective cognition provides multiple perspectives on the same individual, while different artists' drawing styles provide variation in sketch styles. We further have two novel designs to alleviate the challenge of subjectivity. 1) Fusing subjectivity. We propose a non-local (NL) fusion module that gathers sketches from different witnesses for the same identity. 2) Introducing objectivity. An AttrAlign module utilizes attributes as an implicit mask to align cross-domain features. To push forward the advance of Sketch re-ID, we set three benchmarks (large-scale, multi-style, cross-style). Extensive experiments demonstrate our leading performance in these benchmarks. Dataset and Codes are publicly available at: https://github.com/Lin-Kayla/subjectivity-sketch-reid