Beyond Autoregression: Fast LLMs via Self-Distillation Through Time

TL;DR

This paper introduces a novel diffusion-based language model that generates multiple tokens simultaneously, surpasses autoregressive models in quality, and is significantly faster during inference due to a new distillation technique reducing steps by 32-64 times.

Contribution

The paper presents a new discrete diffusion model for language generation that outperforms autoregressive models in speed and quality, with a novel distillation method for efficiency.

Findings

Diffusion models generate at least 32 tokens simultaneously.

Diffusion models outperform AR models in text quality and benchmark performance.

Inference speed is up to 8 times faster than KV-cached AR models at 1.3B parameters.

Abstract

Autoregressive (AR) Large Language Models (LLMs) have demonstrated significant success across numerous tasks. However, the AR modeling paradigm presents certain limitations; for instance, contemporary autoregressive LLMs are trained to generate one token at a time, which can result in noticeable latency. Recent advances have indicated that search and repeated sampling can enhance performance in various applications, such as theorem proving, code generation, and alignment, by utilizing greater computational resources during inference. In this study, we demonstrate that diffusion language models are capable of generating at least 32 tokens simultaneously, while exceeding the performance of AR models in text quality and on the LAMBADA natural language understanding benchmark. This outcome is achieved through a novel distillation method for discrete diffusion models, which reduces the number of inference steps by a factor of 32-64. Practically, at the 1.3B parameters scale, diffusion models, even without caching, can generate tokens at a rate that is up to 8 times faster than AR models employing KV-caching, and we anticipate further improvements with the inclusion of caching. Moreover, we demonstrate the efficacy of our approach for diffusion language models with up to 860M parameters.