GaitSTR: Gait Recognition with Sequential Two-stream Refinement

TL;DR

GaitSTR introduces a two-stream skeleton and silhouette-based gait recognition method that refines skeleton data through self-correction and cross-modal fusion, improving identification accuracy on public datasets.

Contribution

The paper proposes a novel two-stream skeleton and silhouette fusion approach with self- and cross-modal refinement for gait recognition, enhancing accuracy over existing methods.

Findings

Achieves higher recognition accuracy on public datasets.

Refines skeleton data through self-correction in graph convolution.

Demonstrates the effectiveness of multi-modal data fusion.

Abstract

Gait recognition aims to identify a person based on their walking sequences, serving as a useful biometric modality as it can be observed from long distances without requiring cooperation from the subject. In representing a person's walking sequence, silhouettes and skeletons are the two primary modalities used. Silhouette sequences lack detailed part information when overlapping occurs between different body segments and are affected by carried objects and clothing. Skeletons, comprising joints and bones connecting the joints, provide more accurate part information for different segments; however, they are sensitive to occlusions and low-quality images, causing inconsistencies in frame-wise results within a sequence. In this paper, we explore the use of a two-stream representation of skeletons for gait recognition, alongside silhouettes. By fusing the combined data of silhouettes and skeletons, we refine the two-stream skeletons, joints, and bones through self-correction in graph convolution, along with cross-modal correction with temporal consistency from silhouettes. We demonstrate that with refined skeletons, the performance of the gait recognition model can achieve further improvement on public gait recognition datasets compared with state-of-the-art methods without extra annotations.