Defensive Generation

TL;DR

This paper introduces Defensive Generation, an efficient online method for creating outcome indistinguishable generative models that cannot be falsified by a wide range of tests, including higher-order moments.

Contribution

It develops a novel algorithmic framework connecting online multicalibration with variational inequalities and applies it to produce unfalsifiable generative models with optimal convergence rates.

Findings

First to produce outcome indistinguishable models for non-Bernoulli outcomes.

Runs in near-linear time with vanishing T^{-1/2} error rate.

Effective against infinite classes of statistical tests.

Abstract

We study the problem of efficiently producing, in an online fashion, generative models of scalar, multiclass, and vector-valued outcomes that cannot be falsified on the basis of the observed data and a pre-specified collection of computational tests. Our contributions are twofold. First, we expand on connections between online high-dimensional multicalibration with respect to an RKHS and recent advances in expected variational inequality problems, enabling efficient algorithms for the former. We then apply this algorithmic machinery to the problem of outcome indistinguishability. Our procedure, Defensive Generation, is the first to efficiently produce online outcome indistinguishable generative models of non-Bernoulli outcomes that are unfalsifiable with respect to infinite classes of tests, including those that examine higher-order moments of the generated distributions. Furthermore, our method runs in near-linear time in the number of samples and achieves the optimal, vanishing T^{-1/2} rate for generation error.