TCMM: Token Constraint and Multi-Scale Memory Bank of Contrastive Learning for Unsupervised Person Re-identification

TL;DR

This paper introduces TCMM, a novel method combining token constraints and multi-scale memory banks in ViT to improve unsupervised person re-identification by reducing patch noise and enhancing outlier sample utilization.

Contribution

The paper proposes a new ViT Token Constraint and Multi-scale Memory bank to address patch noise and data inconsistency in unsupervised person re-identification.

Findings

Achieves state-of-the-art performance on benchmarks.

Effectively mitigates patch noise in ViT models.

Enhances utilization of outlier samples for robustness.

Abstract

This paper proposes the ViT Token Constraint and Multi-scale Memory bank (TCMM) method to address the patch noises and feature inconsistency in unsupervised person re-identification works. Many excellent methods use ViT features to obtain pseudo labels and clustering prototypes, then train the model with contrastive learning. However, ViT processes images by performing patch embedding, which inevitably introduces noise in patches and may compromise the performance of the re-identification model. On the other hand, previous memory bank based contrastive methods may lead data inconsistency due to the limitation of batch size. Furthermore, existing pseudo label methods often discard outlier samples that are difficult to cluster. It sacrifices the potential value of outlier samples, leading to limited model diversity and robustness. This paper introduces the ViT Token Constraint to mitigate the damage caused by patch noises to the ViT architecture. The proposed Multi-scale Memory enhances the exploration of outlier samples and maintains feature consistency. Experimental results demonstrate that our system achieves state-of-the-art performance on common benchmarks. The project is available at \href{https://github.com/andy412510/TCMM}{https://github.com/andy412510/TCMM}.