Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference

TL;DR

Seed Diffusion is a large-scale diffusion-based language model that achieves high-speed inference through parallel generation, significantly outperforming existing models in speed while maintaining competitive quality on code benchmarks.

Contribution

This paper introduces Seed Diffusion, a novel discrete-state diffusion model that enables fast, parallel inference for language modeling, setting new speed records without sacrificing performance.

Findings

Achieves 2,146 tokens/sec inference speed on H20 GPUs.

Maintains competitive performance across standard code benchmarks.

Outperforms Mercury and Gemini Diffusion in speed while preserving quality.

Abstract

We present Seed Diffusion Preview, a large-scale language model based on discrete-state diffusion, offering remarkably fast inference speed. Thanks to non-sequential, parallel generation, discrete diffusion models provide a notable speedup to mitigate the inherent latency of token-by-token decoding, as demonstrated recently (e.g., Mercury Coder, Gemini Diffusion). Seed Diffusion Preview achieves an inference speed of 2,146 token/s over H20 GPUs while maintaining competitive performance across a sweep of standard code evaluation benchmarks, significantly faster than contemporary Mercury and Gemini Diffusion, establishing new state of the art on the speed-quality Pareto frontier for code models.