Can we spot a fake?

TL;DR

This paper investigates the maximum size of undetectable fake data introduced by an adversary, relating it to the geometric properties of the set of possible adversarial tricks, and extends the analysis beyond Gaussian data.

Contribution

The authors establish bounds on the detectability radius for fake data based on the Gaussian width of the adversary's trick set, generalizing to non-Gaussian distributions and arbitrary sets.

Findings

For symmetric trick sets, the detectability radius is about twice the scaled Gaussian width.

Upper bounds on detectability hold for any set T and distribution of real data.

Conjecture that focusing on the most important directions of T can improve bounds for asymmetric sets.

Abstract

The problem of detecting fake data inspires the following seemingly simple mathematical question. Sample a data point $X$ from the standard normal distribution in $\mathbb{R}^n$. An adversary observes $X$ and corrupts it by adding a vector $rt$, where they can choose any vector $t$ from a fixed set $T$ of the adversary's ``tricks'', and where $r>0$ is a fixed radius. The adversary's choice of $t=t(X)$ may depend on the true data $X$. The adversary wants to hide the corruption by making the fake data $X+rt$ statistically indistinguishable from the real data $X$. What is the largest radius $r=r(T)$ for which the adversary can create an undetectable fake? We show that for highly symmetric sets $T$, the detectability radius $r(T)$ is approximately twice the scaled Gaussian width of $T$. The upper bound actually holds for arbitrary sets $T$ and generalizes to arbitrary, non-Gaussian distributions of real data $X$. The lower bound may fail for not highly symmetric $T$, but we conjecture that this problem can be solved by considering the focused version of the Gaussian width of $T$, which focuses on the most important directions of $T$.