AnCoder: Anchored Code Generation via Discrete Diffusion Models

TL;DR

AnCoder introduces AnchorTree, a structured diffusion framework for code generation that leverages abstract syntax trees to produce syntactically and semantically correct programs more efficiently.

Contribution

The paper presents a novel hierarchical anchoring method using ASTs to improve the quality and efficiency of diffusion-based code generation models.

Findings

AnchorTree effectively guides diffusion models to produce syntactically correct code.

AnCoder achieves high-quality code generation with fewer parameters.

The approach outperforms existing diffusion methods in code accuracy.

Abstract

Diffusion language models offer a compelling alternative to autoregressive code generation, enabling global planning and iterative refinement of complex program logic. However, existing approaches fail to respect the rigid structure of programming languages and, as a result, often produce broken programs that fail to execute. To address this, we introduce AnchorTree, a framework that explicitly anchors the diffusion process using structured, hierarchical priors native to code. Specifically, AnchorTree uses the abstract syntax tree to prioritize resolving syntactically and semantically salient tokens, such as keywords (e.g., if, while) and identifiers (e.g., variable names), thereby establishing a structural scaffold that guides the remaining generation. We validate this framework via AnCoder, a family of models showing that structurally anchored diffusion offers a parameter-efficient path to high-quality code generation.