Find Them All: Unveiling MLLMs for Versatile Person Re-identification

TL;DR

This paper introduces VP-ReID, a comprehensive benchmark for evaluating multi-modal large language models in diverse person re-identification tasks, highlighting their potential and limitations across modalities.

Contribution

The paper presents a new benchmark VP-ReID with extensive multi-modal data and evaluation schemes, and explores the capabilities of MLLMs in person ReID tasks.

Findings

MLLMs show strong versatility and effectiveness in person ReID.

Limitations exist in handling thermal and infrared modalities.

VP-ReID facilitates development of robust cross-modal ReID models.

Abstract

Person re-identification (ReID) aims to retrieve images of a target person from the gallery set, with wide applications in medical rehabilitation and public security. However, traditional person ReID models are typically uni-modal, resulting in limited generalizability across heterogeneous data modalities. Recently, the emergence of multi-modal large language models (MLLMs) has shown a promising avenue for addressing this issue. Despite this potential, existing methods merely regard MLLMs as feature extractors or caption generators, leaving their capabilities in person ReID tasks largely unexplored. To bridge this gap, we introduce a novel benchmark for \underline{\textbf{V}}ersatile \underline{\textbf{P}}erson \underline{\textbf{Re}}-\underline{\textbf{ID}}entification, termed VP-ReID. The benchmark includes 257,310 multi-modal queries and gallery images, covering ten diverse person ReID tasks. In addition, we propose two task-oriented evaluation schemes for MLLM-based person ReID. Extensive experiments demonstrate the impressive versatility, effectiveness, and interpretability of MLLMs in various person ReID tasks. Nevertheless, they also have limitations in handling a few modalities, particularly thermal and infrared data. We hope that VP-ReID can facilitate the community in developing more robust and generalizable cross-modal foundation models for person ReID.