Generalization in Generation: A closer look at Exposure Bias

TL;DR

This paper investigates exposure bias in autoregressive models, emphasizing the importance of generalization, and proposes a new framework combining latent variables and reinforcement learning to improve language modeling.

Contribution

It clarifies the roles of model and learning framework in exposure bias and introduces unconditional generation as a key benchmark for generalization.

Findings

Language modeling results show improved generalization.

Variational auto-encoding confirms model's robustness.

The combined approach effectively handles true and generated contexts.

Abstract

Exposure bias refers to the train-test discrepancy that seemingly arises when an autoregressive generative model uses only ground-truth contexts at training time but generated ones at test time. We separate the contributions of the model and the learning framework to clarify the debate on consequences and review proposed counter-measures. In this light, we argue that generalization is the underlying property to address and propose unconditional generation as its fundamental benchmark. Finally, we combine latent variable modeling with a recent formulation of exploration in reinforcement learning to obtain a rigorous handling of true and generated contexts. Results on language modeling and variational sentence auto-encoding confirm the model's generalization capability.