Multi-Modal Human Authentication Using Silhouettes, Gait and RGB

TL;DR

This paper introduces a dual-modal ensemble approach combining RGB images and silhouettes for improved human authentication, demonstrating superior performance over existing methods in diverse scenarios.

Contribution

The paper proposes DME, a novel method that fuses RGB and silhouette data, and introduces GaitPattern for angle-invariant gait recognition, enhancing robustness.

Findings

Outperforms state-of-the-art recognition systems on CASIA-B dataset

Demonstrates robustness across various viewing angles

Validates effectiveness on newly collected BRIAR dataset

Abstract

Whole-body-based human authentication is a promising approach for remote biometrics scenarios. Current literature focuses on either body recognition based on RGB images or gait recognition based on body shapes and walking patterns; both have their advantages and drawbacks. In this work, we propose Dual-Modal Ensemble (DME), which combines both RGB and silhouette data to achieve more robust performances for indoor and outdoor whole-body based recognition. Within DME, we propose GaitPattern, which is inspired by the double helical gait pattern used in traditional gait analysis. The GaitPattern contributes to robust identification performance over a large range of viewing angles. Extensive experimental results on the CASIA-B dataset demonstrate that the proposed method outperforms state-of-the-art recognition systems. We also provide experimental results using the newly collected BRIAR dataset.