AttriBE: Quantifying Attribute Expressivity in Body Embeddings for Recognition and Identification

TL;DR

This paper introduces a framework to quantify how strongly attributes like BMI, gender, and pose are encoded in transformer-based person re-identification models, revealing attribute hierarchies and their evolution across layers.

Contribution

It extends the concept of expressivity to measure attribute encoding in deep models and applies it to analyze attribute hierarchies in both visible and infrared person re-identification.

Findings

BMI shows highest expressivity in deeper layers.

Attribute encoding varies across layers and training epochs.

Pose becomes more influential in cross-spectral infrared identification.

Abstract

Person re-identification (ReID) systems that match individuals across images or video frames are essential in many real-world applications. However, existing methods are often influenced by attributes such as gender, pose, and body mass index (BMI), which vary in unconstrained settings and raise concerns related to fairness and generalization. To address this, we extend the notion of expressivity, defined as the mutual information between learned features and specific attributes, using a secondary neural network to quantify how strongly attributes are encoded. Applying this framework to three transformer-based ReID models on a large-scale visible-spectrum dataset, we find that BMI consistently shows the highest expressivity in deeper layers. Attributes in the final representation are ranked as BMI > Pitch > Gender > Yaw, and expressivity evolves across layers and training epochs, with pose peaking in intermediate layers and BMI strengthening with depth. We further extend the analysis to cross-spectral person identification across infrared modalities including short-wave, medium-wave, and long-wave infrared. In this setting, pitch becomes comparable to BMI and attribute trends increase monotonically across depth, suggesting increased reliance on structural cues when bridging modality gaps. Overall, the results show that transformer-based ReID embeddings encode a hierarchy of implicit attributes, with morphometric information persistently embedded and pose contributing more strongly under cross-spectral conditions.