MMGait: Towards Multi-Modal Gait Recognition

TL;DR

MMGait introduces a comprehensive multi-modal gait recognition benchmark with diverse sensors and modalities, enabling systematic evaluation and a new unified recognition task, supported by a baseline model and publicly available resources.

Contribution

The paper presents MMGait, a large-scale multi-modal gait dataset and benchmark, along with a new unified recognition task and a baseline model for cross-modal gait recognition.

Findings

MMGait contains 334,060 sequences from 725 subjects across five sensor modalities.

Extensive evaluations reveal modality robustness and complementarity in gait recognition.

The proposed OmniGait baseline effectively learns shared embeddings across diverse modalities.

Abstract

Gait recognition has emerged as a powerful biometric technique for identifying individuals at a distance without requiring user cooperation. Most existing methods focus primarily on RGB-derived modalities, which fall short in real-world scenarios requiring multi-modal collaboration and cross-modal retrieval. To overcome these challenges, we present MMGait, a comprehensive multi-modal gait benchmark integrating data from five heterogeneous sensors, including an RGB camera, a depth camera, an infrared camera, a LiDAR scanner, and a 4D Radar system. MMGait contains twelve modalities and 334,060 sequences from 725 subjects, enabling systematic exploration across geometric, photometric, and motion domains. Based on MMGait, we conduct extensive evaluations on single-modal, cross-modal, and multi-modal paradigms to analyze modality robustness and complementarity. Furthermore, we introduce a new task, Omni Multi-Modal Gait Recognition, which aims to unify the above three gait recognition paradigms within a single model. We also propose a simple yet powerful baseline, OmniGait, which learns a shared embedding space across diverse modalities and achieves promising recognition performance. The MMGait benchmark, codebase, and pretrained checkpoints are publicly available at https://github.com/BNU-IVC/MMGait.