Not All Starting Points Are Equal: Pre-trained Priors and Their Outsized Impact on Person Identification

TL;DR

This paper investigates how different pre-trained models significantly influence person re-identification performance, emphasizing the importance of pre-trained priors in domain adaptation and achieving state-of-the-art results.

Contribution

It introduces a framework viewing pre-trained weights as priors, demonstrating their impact on re-id outcomes and establishing simple fine-tuning as a strong baseline.

Findings

Pre-trained models serve as strong priors influencing re-id performance.

Large foundation models with simple adaptation achieve SOTA results.

Performance is sensitive to optimizer, weight-decay, and loss function choices.

Abstract

Recent years have seen an explosion of diverse general purpose pre-training methodologies for computer vision. However, the impact that these pre-training methodologies have on person identification tasks (re-id) remains under-explored. We show that under equated domain adaptation pipelines, there is dramatic variance in person identification outcomes using different starting models (architectures and pre-trained weights). We show that a range of intuitive explanations for differing downstream performance on a range of re-id tests are insufficient and propose that pre-trained weights serve as a strong prior to the weights learned during domain adaptation. This framework allows for domain adapted solutions to be viewed as a maximum probability point estimate of the Gibbs posterior with the pre-trained weights acting as a prior. Under this framework, we show that large, pre-trained foundation models with simple domain adaptation achieve SOTA solutions on a range of re-id datasets (Market, PRCC, DeepChange, BTS) with solutions that are very close in the parameter space to the starting parameters. Moreover, we perform ablations on these solutions and show that they can be reached with small transfer sets and with varying transfer datasets but are sensitive to choice of optimizer, weight-decay, and loss function. Ultimately, we propose that the simple approach of direct fine-tuning using large vision foundation models (CLIP, Dino, EVA, AIM, etc.) needs to serve as an important baseline for future work in re-id.