DepthGait: Multi-Scale Cross-Level Feature Fusion of RGB-Derived Depth and Silhouette Sequences for Robust Gait Recognition

TL;DR

DepthGait introduces a multi-modal gait recognition framework combining RGB-derived depth maps and silhouettes, employing a novel fusion scheme to improve robustness against viewpoint variations and achieve state-of-the-art accuracy.

Contribution

The paper presents a new multi-scale cross-level fusion method for integrating depth and silhouette modalities in gait recognition, enhancing discriminative feature extraction.

Findings

Achieves state-of-the-art accuracy on standard benchmarks.

Effectively handles viewpoint variations in gait recognition.

Demonstrates the benefit of combining depth and silhouette modalities.

Abstract

Robust gait recognition requires highly discriminative representations, which are closely tied to input modalities. While binary silhouettes and skeletons have dominated recent literature, these 2D representations fall short of capturing sufficient cues that can be exploited to handle viewpoint variations, and capture finer and meaningful details of gait. In this paper, we introduce a novel framework, termed DepthGait, that incorporates RGB-derived depth maps and silhouettes for enhanced gait recognition. Specifically, apart from the 2D silhouette representation of the human body, the proposed pipeline explicitly estimates depth maps from a given RGB image sequence and uses them as a new modality to capture discriminative features inherent in human locomotion. In addition, a novel multi-scale and cross-level fusion scheme has also been developed to bridge the modality gap between depth maps and silhouettes. Extensive experiments on standard benchmarks demonstrate that the proposed DepthGait achieves state-of-the-art performance compared to peer methods and attains an impressive mean rank-1 accuracy on the challenging datasets.