Inverting Black-Box Face Recognition Systems via Zero-Order Optimization in Eigenface Space

TL;DR

This paper presents DarkerBB, a zero-order optimization method that reconstructs color facial images from black-box recognition models using only similarity scores, highlighting privacy risks.

Contribution

DarkerBB is the first approach to perform face reconstruction solely from similarity scores in eigenface space, achieving state-of-the-art results.

Findings

Achieves high verification accuracy on multiple benchmarks

Operates efficiently with limited similarity score information

Demonstrates privacy risks in black-box face recognition systems

Abstract

Reconstructing facial images from black-box recognition models poses a significant privacy threat. While many methods require access to embeddings, we address the more challenging scenario of model inversion using only similarity scores. This paper introduces DarkerBB, a novel approach that reconstructs color faces by performing zero-order optimization within a PCA-derived eigenface space. Despite this highly limited information, experiments on LFW, AgeDB-30, and CFP-FP benchmarks demonstrate that DarkerBB achieves state-of-the-art verification accuracies in the similarity-only setting, with competitive query efficiency.