Test-Time Augmentation for Pose-invariant Face Recognition

TL;DR

This paper introduces Pose-TTA, a test-time augmentation method that improves face recognition by aligning and synthesizing pose-matched images without retraining models, leading to more robust identity verification.

Contribution

Pose-TTA is a novel, training-free approach that aligns faces at inference time using a portrait animator, generating pose-matched images to enhance recognition accuracy.

Findings

Consistently improves face recognition performance across datasets.

No retraining or fine-tuning needed for existing models.

Effective in reducing pose-related identity loss.

Abstract

The goal of this paper is to enhance face recognition performance by augmenting head poses during the testing phase. Existing methods often rely on training on frontalised images or learning pose-invariant representations, yet both approaches typically require re-training and testing for each dataset, involving a substantial amount of effort. In contrast, this study proposes Pose-TTA, a novel approach that aligns faces at inference time without additional training. To achieve this, we employ a portrait animator that transfers the source image identity into the pose of a driving image. Instead of frontalising a side-profile face -- which can introduce distortion -- Pose-TTA generates matching side-profile images for comparison, thereby reducing identity information loss. Furthermore, we propose a weighted feature aggregation strategy to address any distortions or biases arising from the synthetic data, thus enhancing the reliability of the augmented images. Extensive experiments on diverse datasets and with various pre-trained face recognition models demonstrate that Pose-TTA consistently improves inference performance. Moreover, our method is straightforward to integrate into existing face recognition pipelines, as it requires no retraining or fine-tuning of the underlying recognition models.