Incorporating Reinforced Adversarial Learning in Autoregressive Image Generation

TL;DR

This paper introduces Reinforced Adversarial Learning (RAL) for autoregressive image models, enhancing visual fidelity and training stability by combining adversarial loss with policy gradient optimization, achieving state-of-the-art results.

Contribution

It is the first to incorporate adversarial learning into autoregressive models for image generation, addressing exposure bias and improving visual quality.

Findings

Improved negative log-likelihood (NLL) and FID scores.

Achieved state-of-the-art results on CelebA 64x64 images.

Enhanced training stability and diversity in generated images.

Abstract

Autoregressive models recently achieved comparable results versus state-of-the-art Generative Adversarial Networks (GANs) with the help of Vector Quantized Variational AutoEncoders (VQ-VAE). However, autoregressive models have several limitations such as exposure bias and their training objective does not guarantee visual fidelity. To address these limitations, we propose to use Reinforced Adversarial Learning (RAL) based on policy gradient optimization for autoregressive models. By applying RAL, we enable a similar process for training and testing to address the exposure bias issue. In addition, visual fidelity has been further optimized with adversarial loss inspired by their strong counterparts: GANs. Due to the slow sampling speed of autoregressive models, we propose to use partial generation for faster training. RAL also empowers the collaboration between different modules of the VQ-VAE framework. To our best knowledge, the proposed method is first to enable adversarial learning in autoregressive models for image generation. Experiments on synthetic and real-world datasets show improvements over the MLE trained models. The proposed method improves both negative log-likelihood (NLL) and Fr\'echet Inception Distance (FID), which indicates improvements in terms of visual quality and diversity. The proposed method achieves state-of-the-art results on Celeba for 64 $\times$ 64 image resolution, showing promise for large scale image generation.