What really matters for person re-identification? A Mixture-of-Experts Framework for Semantic Attribute Importance

TL;DR

This paper introduces MoSAIC-ReID, a framework that systematically quantifies the importance of semantic attributes in person re-identification models, providing insights into which attributes most influence model decisions.

Contribution

It presents a novel Mixture-of-Experts framework with an oracle router for attribute attribution analysis in person re-identification, enabling large-scale interpretability.

Findings

Clothing colors and intrinsic features are most influential attributes.

Accessory cues have limited impact on re-identification accuracy.

MoSAIC-ReID achieves competitive performance while offering interpretability.

Abstract

State-of-the-art person re-identification methods achieve impressive accuracy but remain largely opaque, leaving open the question: which high-level semantic attributes do these models actually rely on? We propose MoSAIC-ReID, a Mixture-of-Experts framework that systematically quantifies the importance of pedestrian attributes for re-identification. Our approach uses LoRA-based experts, each linked to a single attribute, and an oracle router that enables controlled attribution analysis. While MoSAIC-ReID achieves competitive performance on Market-1501 and DukeMTMC under the assumption that attribute annotations are available at test time, its primary value lies in providing a large-scale, quantitative study of attribute importance across intrinsic and extrinsic cues. Using generalized linear models, statistical tests, and feature-importance analyses, we reveal which attributes, such as clothing colors and intrinsic characteristics, contribute most strongly, while infrequent cues (e.g. accessories) have limited effect. This work offers a principled framework for interpretable ReID and highlights the requirements for integrating explicit semantic knowledge in practice. Code is available at https://github.com/psaltaath/MoSAIC-ReID