D5P4: Partition Determinantal Point Process for Diversity in Parallel Discrete Diffusion Decoding

TL;DR

This paper introduces D5P4, a novel diversity-enhanced decoding method for discrete diffusion models that improves output variety without significant computational cost.

Contribution

It presents a generalized beam-search framework for discrete diffusion decoding and formulates diversity control as MAP inference over a Determinantal Point Process.

Findings

D5P4 enhances diversity in generated outputs.

Maintains competitive quality with existing methods.

Operates efficiently on multi-GPU systems.

Abstract

Discrete diffusion models are promising alternatives to autoregressive approaches for text generation, yet their decoding methods remain under-studied. Standard decoding methods for autoregressive models, such as beam search, do not directly apply to iterative denoising, and existing diffusion decoding techniques provide limited control over in-batch diversity. To bridge this gap, we introduce a generalized beam-search framework for discrete diffusion that generates candidates in parallel and supports modular beam-selection objectives. As a diversity-focused instantiation, we propose D5P4, which formulates the selection step as MAP inference over a Determinantal Point Process. Leveraging a scalable greedy solver, D5P4 maintains multi-GPU compatibility and enables an explicit trade-off between model probability and target diversity with near-zero compute overhead. Experiments on free-form generation and question answering demonstrate that D5P4 improves diversity over strong baselines while maintaining competitive generation quality.