A Characteristic Function Approach to Deep Implicit Generative Modeling

TL;DR

This paper introduces a new characteristic function distance (CFD) for training implicit generative models, offering a computationally efficient, stable, and theoretically sound alternative to existing methods like GANs and MMD-GANs.

Contribution

It formulates a novel CFD metric for implicit generative modeling, with a scalable computation method and an adaptive variant that enhances training and sample quality.

Findings

CFD is computationally efficient with linear complexity.

The proposed method outperforms WGAN and MMD-GAN on image benchmarks.

The adaptive CFD improves test power and reduces manual tuning.

Abstract

Implicit Generative Models (IGMs) such as GANs have emerged as effective data-driven models for generating samples, particularly images. In this paper, we formulate the problem of learning an IGM as minimizing the expected distance between characteristic functions. Specifically, we minimize the distance between characteristic functions of the real and generated data distributions under a suitably-chosen weighting distribution. This distance metric, which we term as the characteristic function distance (CFD), can be (approximately) computed with linear time-complexity in the number of samples, in contrast with the quadratic-time Maximum Mean Discrepancy (MMD). By replacing the discrepancy measure in the critic of a GAN with the CFD, we obtain a model that is simple to implement and stable to train. The proposed metric enjoys desirable theoretical properties including continuity and differentiability with respect to generator parameters, and continuity in the weak topology. We further propose a variation of the CFD in which the weighting distribution parameters are also optimized during training; this obviates the need for manual tuning, and leads to an improvement in test power relative to CFD. We demonstrate experimentally that our proposed method outperforms WGAN and MMD-GAN variants on a variety of unsupervised image generation benchmarks.