HAMoBE: Hierarchical and Adaptive Mixture of Biometric Experts for Video-based Person ReID

TL;DR

HAMoBE introduces a hierarchical, adaptive framework that models key biometric features for video-based person re-identification, significantly improving matching accuracy by dynamically integrating appearance, shape, and gait cues.

Contribution

It proposes a novel multi-level biometric expert system with a dual-input gating network, leveraging multi-layer features from large pre-trained models for enhanced ReID performance.

Findings

Achieves +13.0% Rank-1 accuracy on MEVID benchmark.

Effectively models appearance, shape, and gait features independently.

Demonstrates robustness across varied video scenarios.

Abstract

Recently, research interest in person re-identification (ReID) has increasingly focused on video-based scenarios, which are essential for robust surveillance and security in varied and dynamic environments. However, existing video-based ReID methods often overlook the necessity of identifying and selecting the most discriminative features from both videos in a query-gallery pair for effective matching. To address this issue, we propose a novel Hierarchical and Adaptive Mixture of Biometric Experts (HAMoBE) framework, which leverages multi-layer features from a pre-trained large model (e.g., CLIP) and is designed to mimic human perceptual mechanisms by independently modeling key biometric features--appearance, static body shape, and dynamic gait--and adaptively integrating them. Specifically, HAMoBE includes two levels: the first level extracts low-level features from multi-layer representations provided by the frozen large model, while the second level consists of specialized experts focusing on long-term, short-term, and temporal features. To ensure robust matching, we introduce a new dual-input decision gating network that dynamically adjusts the contributions of each expert based on their relevance to the input scenarios. Extensive evaluations on benchmarks like MEVID demonstrate that our approach yields significant performance improvements (e.g., +13.0% Rank-1 accuracy).