Diffusion Language Model Inference with Monte Carlo Tree Search

TL;DR

This paper introduces MEDAL, a Monte Carlo Tree Search-based inference framework for diffusion language models, significantly improving text generation quality without extra training by exploring promising unmasking paths.

Contribution

The paper presents MEDAL, a novel inference-time scaling method using Monte Carlo Tree Search for diffusion language models, enabling better decoding paths without additional training.

Findings

Up to 22.0% improvement over existing inference methods

Effective exploration of unmasking trajectories with MCTS

Enhances generation quality as search budget increases

Abstract

Diffusion language models (DLMs) have recently emerged as a compelling alternative to autoregressive generation, offering parallel generation and improved global coherence. During inference, DLMs generate text by iteratively denoising masked sequences in parallel; however, determining which positions to unmask and which tokens to commit forms a large combinatorial search problem. Existing inference methods approximate this search using heuristics, which often yield suboptimal decoding paths; other approaches instead rely on additional training to guide token selection. To introduce a principled search mechanism for DLMs inference, we introduce MEDAL, an inference-time scaling framework that integrates Monte Carlo Tree SEarch initialization for Diffusion LAnguage Model inference. We employ Monte Carlo Tree Search at the initialization stage to explore promising unmasking trajectories, providing a robust starting point for subsequent refinement. This design enables efficient inference-time scaling, allowing generation quality to improve as the search budget increases, without additional training. Across multiple benchmarks, MEDAL achieves up to 22.0% improvement over existing inference strategies, establishing a new paradigm for search-based inference in DLMs.