Adaptive Divergence for Rapid Adversarial Optimization

TL;DR

This paper introduces a new family of divergences for adversarial optimization that adaptively vary model capacity, significantly reducing computational costs in tasks like tuning physics simulators.

Contribution

It proposes a novel divergence family that adjusts model capacity during adversarial optimization to accelerate convergence and reduce computational costs.

Findings

Effective in tuning a physics simulator, Pythia.

Reduces computational costs compared to traditional methods.

Demonstrates faster convergence in adversarial tasks.

Abstract

Adversarial Optimization (AO) provides a reliable, practical way to match two implicitly defined distributions, one of which is usually represented by a sample of real data, and the other is defined by a generator. Typically, AO involves training of a high-capacity model on each step of the optimization. In this work, we consider computationally heavy generators, for which training of high-capacity models is associated with substantial computational costs. To address this problem, we introduce a novel family of divergences, which varies the capacity of the underlying model, and allows for a significant acceleration with respect to the number of samples drawn from the generator. We demonstrate the performance of the proposed divergences on several tasks, including tuning parameters of a physics simulator, namely, Pythia event generator.