ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding

TL;DR

ReFusion is a novel masked diffusion model that enhances parallel decoding in large language models by integrating sequence reorganization, leading to significant speedups and performance improvements over prior diffusion models while approaching autoregressive model quality.

Contribution

ReFusion introduces a slot-level diffusion approach with sequence reorganization, enabling full KV cache reuse and reducing learning complexity, thus improving speed and performance of language models.

Findings

34% performance gain over prior diffusion models

Over 18× average speedup compared to previous MDMs

Bridges the performance gap to autoregressive models

Abstract

Autoregressive models (ARMs) are hindered by slow sequential inference. While masked diffusion models (MDMs) offer a parallel alternative, they suffer from critical drawbacks: high computational overhead from precluding Key-Value (KV) caching, and incoherent generation arising from learning dependencies over an intractable space of token combinations. To address these limitations, we introduce \textsc{ReFusion}, a novel masked diffusion model that integrates sequence reorganization into the causal attention framework. By elevating parallel decoding from the token level to a higher slot level, \textsc{ReFusion} interleaves inter-slot diffusion-based selection with intra-slot autoregressive infilling, while reordering newly generated slots ahead of the remaining masks after each iteration. Consequently, this design simultaneously unlocks full KV cache reuse and reduces learning complexity from an intractable token combination space to a manageable slot-level permutation space. Extensive experiments on seven diverse benchmarks show that \textsc{ReFusion} not only overwhelmingly surpasses prior MDMs with a 34\% performance gain and an over 18$\times$ speedup on average, but also bridges the performance gap to strong ARMs while maintaining a 2.33$\times$ average speedup.