Discrete Flow Maps
Peter Potaptchik, Jason Yim, Adhi Saravanan, Peter Holderrieth, Eric Vanden-Eijnden, Michael S. Albergo

TL;DR
Discrete Flow Maps introduce a novel framework for sequence generation that compresses trajectories into single-step mappings, aligning with the geometry of discrete data to improve language modeling efficiency.
Contribution
The paper presents Discrete Flow Maps, a new approach that adapts flow models to discrete data, overcoming geometric and training challenges for improved language sequence generation.
Findings
Outperforms previous state-of-the-art in discrete flow modeling.
Reconciles trajectory compression with the geometry of the probability simplex.
Enables single-pass sequence generation from noise.
Abstract
The sequential nature of autoregressive next-token prediction imposes a fundamental speed limit on large language models. While continuous flow models offer a path to parallel generation, they traditionally demand expensive iterative integration. Flow Maps bypass this bottleneck by compressing generative trajectories into single-step mappings, theoretically enabling the generation of full text sequences from noise in a single forward pass. However, standard formulations rely on Euclidean regression losses that are geometrically ill-suited for discrete data. In this work, we resolve this conflict with Discrete Flow Maps, a framework that reconciles trajectory compression with the geometry of the probability simplex. We recast standard flow map training for the discrete domain, aligning the training dynamics with the discrete nature of language. Empirically, this strict geometric…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
