Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning

TL;DR

This paper introduces a novel reasoning framework combining diffusion language models for efficient thought proposal with large language models for evaluation, significantly improving reasoning efficiency and accuracy.

Contribution

The work pioneers using diffusion language models to generate intermediate reasoning steps, reducing computational costs compared to autoregressive methods.

Findings

Achieves strong performance on complex reasoning benchmarks.

Reduces computational overhead in reasoning tasks.

Demonstrates effective collaboration between DLMs and LLMs.

Abstract

In recent years, large language models (LLMs) have witnessed remarkable advancements, with the test-time scaling law consistently enhancing the reasoning capabilities. Through systematic evaluation and exploration of a diverse spectrum of intermediate thoughts, LLMs demonstrate the potential to generate deliberate reasoning steps, thereby substantially enhancing reasoning accuracy. However, LLMs' autoregressive generation paradigm results in reasoning performance scaling sub-optimally with test-time computation, often requiring excessive computational overhead to propose thoughts while yielding only marginal performance gains. In contrast, diffusion language models (DLMs) can efficiently produce diverse samples through parallel denoising in a single forward pass, inspiring us to leverage them for proposing intermediate thoughts, thereby alleviating the computational burden associated with autoregressive generation while maintaining quality. In this work, we propose an efficient collaborative reasoning framework, leveraging DLMs to generate candidate thoughts and LLMs to evaluate their quality. Experiments across diverse benchmarks demonstrate that our framework achieves strong performance in complex reasoning tasks, offering a promising direction for future research. Our code is open-source at https://anonymous.4open.science/r/Diffuse-Thinking-EC60.