Curriculum-Based Neighborhood Sampling For Sequence Prediction

TL;DR

This paper introduces a curriculum learning approach called Nearest-Neighbor Replacement Sampling to reduce exposure bias in language models, improving multi-step prediction accuracy by gradually introducing stochasticity during training.

Contribution

It proposes a novel curriculum learning method that replaces inputs with similar neighbors to better handle exposure bias in sequence prediction models.

Findings

Improves performance on language modeling benchmarks.

Works well with scheduled sampling to reduce compounding errors.

Requires minimal additional memory.

Abstract

The task of multi-step ahead prediction in language models is challenging considering the discrepancy between training and testing. At test time, a language model is required to make predictions given past predictions as input, instead of the past targets that are provided during training. This difference, known as exposure bias, can lead to the compounding of errors along a generated sequence at test time. In order to improve generalization in neural language models and address compounding errors, we propose a curriculum learning based method that gradually changes an initially deterministic teacher policy to a gradually more stochastic policy, which we refer to as \textit{Nearest-Neighbor Replacement Sampling}. A chosen input at a given timestep is replaced with a sampled nearest neighbor of the past target with a truncated probability proportional to the cosine similarity between the original word and its top $k$ most similar words. This allows the teacher to explore alternatives when the teacher provides a sub-optimal policy or when the initial policy is difficult for the learner to model. The proposed strategy is straightforward, online and requires little additional memory requirements. We report our main findings on two language modelling benchmarks and find that the proposed approach performs particularly well when used in conjunction with scheduled sampling, that too attempts to mitigate compounding errors in language models.