Prediction Exposes Your Face: Black-box Model Inversion via Prediction Alignment

TL;DR

This paper introduces a novel black-box model inversion attack called Prediction-to-Image (P2I) that efficiently reconstructs private facial images from model predictions by aligning prediction vectors with a disentangled latent space, significantly reducing queries.

Contribution

The paper presents a new P2I method with a Prediction Alignment Encoder and an Aligned Ensemble Attack, enabling high-accuracy image reconstruction with fewer queries in black-box settings.

Findings

Outperforms state-of-the-art methods in attack accuracy.

Reduces query numbers by 99% compared to previous approaches.

Achieves 8.5% higher attack accuracy on CelebA dataset.

Abstract

Model inversion (MI) attack reconstructs the private training data of a target model given its output, posing a significant threat to deep learning models and data privacy. On one hand, most of existing MI methods focus on searching for latent codes to represent the target identity, yet this iterative optimization-based scheme consumes a huge number of queries to the target model, making it unrealistic especially in black-box scenario. On the other hand, some training-based methods launch an attack through a single forward inference, whereas failing to directly learn high-level mappings from prediction vectors to images. Addressing these limitations, we propose a novel Prediction-to-Image (P2I) method for black-box MI attack. Specifically, we introduce the Prediction Alignment Encoder to map the target model's output prediction into the latent code of StyleGAN. In this way, prediction vector space can be well aligned with the more disentangled latent space, thus establishing a connection between prediction vectors and the semantic facial features. During the attack phase, we further design the Aligned Ensemble Attack scheme to integrate complementary facial attributes of target identity for better reconstruction. Experimental results show that our method outperforms other SOTAs, e.g.,compared with RLB-MI, our method improves attack accuracy by 8.5% and reduces query numbers by 99% on dataset CelebA.