Privacy Attacks on Image AutoRegressive Models

TL;DR

This paper reveals that Image AutoRegressive models, despite high image quality and speed, pose significant privacy risks, including high success rates in membership inference and data extraction, surpassing diffusion models.

Contribution

It introduces a novel membership inference attack tailored for IARs, demonstrating their higher vulnerability to privacy breaches compared to diffusion models.

Findings

IARs have a 94.57% success rate in membership inference attacks.

As few as 4 samples are needed for dataset inference in IARs.

Hundreds of training data points can be extracted from an IAR.

Abstract

Image AutoRegressive generation has emerged as a new powerful paradigm with image autoregressive models (IARs) matching state-of-the-art diffusion models (DMs) in image quality (FID: 1.48 vs. 1.58) while allowing for a higher generation speed. However, the privacy risks associated with IARs remain unexplored, raising concerns regarding their responsible deployment. To address this gap, we conduct a comprehensive privacy analysis of IARs, comparing their privacy risks to the ones of DMs as reference points. Concretely, we develop a novel membership inference attack (MIA) that achieves a remarkably high success rate in detecting training images (with True Positive Rate at False Positive Rate = 1% of 94.57% vs. 6.38% for DMs with comparable attacks). We leverage our novel MIA to provide dataset inference (DI) for IARs, and show that it requires as few as 4 samples to detect dataset membership (compared to 200 for DI in DMs), confirming a higher information leakage in IARs. Finally, we are able to extract hundreds of training data points from an IAR (e.g., 698 from VAR-\textit{d}30). Our results suggest a fundamental privacy-utility trade-off: while IARs excel in image generation quality and speed, they are \textit{empirically} significantly more vulnerable to privacy attacks compared to DMs that achieve similar performance. We release the code at https://github.com/sprintml/privacy_attacks_against_iars for reproducibility.