Exploring 3D Face Reconstruction and Fusion Methods for Face Verification: A Case-Study in Video Surveillance

TL;DR

This study evaluates three state-of-the-art 3D face reconstruction algorithms for face verification in video surveillance, demonstrating that combining their outputs enhances accuracy across varying distances and camera types.

Contribution

It introduces a fusion approach using multiple 3DFR algorithms to improve face verification performance in challenging surveillance conditions.

Findings

Fusion of different 3DFR algorithms improves verification accuracy.

Performance gains are significant in cross-distance and cross-camera scenarios.

Using multiple algorithms enhances robustness in real-world surveillance settings.

Abstract

3D face reconstruction (3DFR) algorithms are based on specific assumptions tailored to distinct application scenarios. These assumptions limit their use when acquisition conditions, such as the subject's distance from the camera or the camera's characteristics, are different than expected, as typically happens in video surveillance. Additionally, 3DFR algorithms follow various strategies to address the reconstruction of a 3D shape from 2D data, such as statistical model fitting, photometric stereo, or deep learning. In the present study, we explore the application of three 3DFR algorithms representative of the SOTA, employing each one as the template set generator for a face verification system. The scores provided by each system are combined by score-level fusion. We show that the complementarity induced by different 3DFR algorithms improves performance when tests are conducted at never-seen-before distances from the camera and camera characteristics (cross-distance and cross-camera settings), thus encouraging further investigations on multiple 3DFR-based approaches.