Gait Recognition using Multi-Scale Partial Representation Transformation with Capsules

TL;DR

This paper introduces a novel deep learning framework combining multi-scale partial gait representations, recurrent correlation learning, and capsule networks to improve gait recognition robustness against viewpoint and appearance variations.

Contribution

It proposes a new deep network architecture that leverages capsules and recurrent learning to enhance gait feature discrimination under challenging conditions.

Findings

Outperforms state-of-the-art on CASIA-B and OU-MVLP datasets.

Robust to viewpoint and appearance changes.

Effective in challenging carrying and viewing scenarios.

Abstract

Gait recognition, referring to the identification of individuals based on the manner in which they walk, can be very challenging due to the variations in the viewpoint of the camera and the appearance of individuals. Current methods for gait recognition have been dominated by deep learning models, notably those based on partial feature representations. In this context, we propose a novel deep network, learning to transfer multi-scale partial gait representations using capsules to obtain more discriminative gait features. Our network first obtains multi-scale partial representations using a state-of-the-art deep partial feature extractor. It then recurrently learns the correlations and co-occurrences of the patterns among the partial features in forward and backward directions using Bi-directional Gated Recurrent Units (BGRU). Finally, a capsule network is adopted to learn deeper part-whole relationships and assigns more weights to the more relevant features while ignoring the spurious dimensions. That way, we obtain final features that are more robust to both viewing and appearance changes. The performance of our method has been extensively tested on two gait recognition datasets, CASIA-B and OU-MVLP, using four challenging test protocols. The results of our method have been compared to the state-of-the-art gait recognition solutions, showing the superiority of our model, notably when facing challenging viewing and carrying conditions.