Robust Face Verification via Disentangled Representations

TL;DR

This paper proposes a novel face verification method using disentangled generative models and contrastive learning, improving robustness against adversarial attacks by training with hard negative pairs.

Contribution

It introduces a new approach combining disentangled generative models with contrastive loss for robust face verification, leveraging online augmentation during training.

Findings

Higher clean and robust accuracy compared to state-of-the-art methods.

Effective against white-box physical adversarial attacks.

Efficient convergence with a weak inner solver.

Abstract

We introduce a robust algorithm for face verification, i.e., deciding whether twoimages are of the same person or not. Our approach is a novel take on the idea ofusing deep generative networks for adversarial robustness. We use the generativemodel during training as an online augmentation method instead of a test-timepurifier that removes adversarial noise. Our architecture uses a contrastive loss termand a disentangled generative model to sample negative pairs. Instead of randomlypairing two real images, we pair an image with its class-modified counterpart whilekeeping its content (pose, head tilt, hair, etc.) intact. This enables us to efficientlysample hard negative pairs for the contrastive loss. We experimentally show that, when coupled with adversarial training, the proposed scheme converges with aweak inner solver and has a higher clean and robust accuracy than state-of-the-art-methods when evaluated against white-box physical attacks.