AAformer: Auto-Aligned Transformer for Person Re-Identification

TL;DR

The paper introduces AAformer, a transformer-based model that automatically locates and extracts fine-grained part and nonpart features for person re-identification, outperforming existing CNN-based methods.

Contribution

It proposes the auto-aligned transformer (AAformer) with learnable part tokens and an auto-alignment mechanism using optimal transport for precise part localization.

Findings

AAformer outperforms state-of-the-art methods in person re-ID.

The part tokens effectively capture fine-grained features.

Auto-alignment improves patch grouping accuracy.

Abstract

In person re-identification (re-ID), extracting part-level features from person images has been verified to be crucial to offer fine-grained information. Most of the existing CNN-based methods only locate the human parts coarsely, or rely on pretrained human parsing models and fail in locating the identifiable nonhuman parts (e.g., knapsack). In this article, we introduce an alignment scheme in transformer architecture for the first time and propose the auto-aligned transformer (AAformer) to automatically locate both the human parts and nonhuman ones at patch level. We introduce the "Part tokens ([PART]s)", which are learnable vectors, to extract part features in the transformer. A [PART] only interacts with a local subset of patches in self-attention and learns to be the part representation. To adaptively group the image patches into different subsets, we design the auto-alignment. Auto-alignment employs a fast variant of optimal transport (OT) algorithm to online cluster the patch embeddings into several groups with the [PART]s as their prototypes. AAformer integrates the part alignment into the self-attention and the output [PART]s can be directly used as part features for retrieval. Extensive experiments validate the effectiveness of [PART]s and the superiority of AAformer over various state-of-the-art methods.