Adversarial score matching and improved sampling for image generation

TL;DR

This paper enhances score matching-based image generation by introducing improved sampling techniques and a hybrid training approach, achieving results competitive with leading methods like GANs on CIFAR-10.

Contribution

It proposes two novel improvements to denoising score matching: consistent annealed sampling and a hybrid training formulation combining score matching and adversarial objectives.

Findings

Achieves competitive image quality on CIFAR-10

Introduces stable sampling method for score-based models

Demonstrates effectiveness of hybrid training approach

Abstract

Denoising Score Matching with Annealed Langevin Sampling (DSM-ALS) has recently found success in generative modeling. The approach works by first training a neural network to estimate the score of a distribution, and then using Langevin dynamics to sample from the data distribution assumed by the score network. Despite the convincing visual quality of samples, this method appears to perform worse than Generative Adversarial Networks (GANs) under the Fr\'echet Inception Distance, a standard metric for generative models. We show that this apparent gap vanishes when denoising the final Langevin samples using the score network. In addition, we propose two improvements to DSM-ALS: 1) Consistent Annealed Sampling as a more stable alternative to Annealed Langevin Sampling, and 2) a hybrid training formulation, composed of both Denoising Score Matching and adversarial objectives. By combining these two techniques and exploring different network architectures, we elevate score matching methods and obtain results competitive with state-of-the-art image generation on CIFAR-10.