Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes

TL;DR

This paper introduces a bimodal gait recognition network that combines skeleton and silhouette data, utilizing a novel multi-scale gait graph to improve identification accuracy across challenging conditions.

Contribution

It proposes a new bimodal fusion network with a multi-scale gait graph for enhanced gait recognition by effectively integrating skeleton and silhouette features.

Findings

Achieves 92.1% rank-1 accuracy on CASIA-B in challenging conditions.

Demonstrates superiority of MSGG in modeling skeletons.

Shows effectiveness of bimodal fusion in gait recognition.

Abstract

Gait recognition captures gait patterns from the walking sequence of an individual for identification. Most existing gait recognition methods learn features from silhouettes or skeletons for the robustness to clothing, carrying, and other exterior factors. The combination of the two data modalities, however, is not fully exploited. Previous multimodal gait recognition methods mainly employ the skeleton to assist the local feature extraction where the intrinsic discrimination of the skeleton data is ignored. This paper proposes a simple yet effective Bimodal Fusion (BiFusion) network which mines discriminative gait patterns in skeletons and integrates with silhouette representations to learn rich features for identification. Particularly, the inherent hierarchical semantics of body joints in a skeleton is leveraged to design a novel Multi-Scale Gait Graph (MSGG) network for the feature extraction of skeletons. Extensive experiments on CASIA-B and OUMVLP demonstrate both the superiority of the proposed MSGG network in modeling skeletons and the effectiveness of the bimodal fusion for gait recognition. Under the most challenging condition of walking in different clothes on CASIA-B, our method achieves the rank-1 accuracy of 92.1%.