Score-Based Generative Modeling through Anisotropic Stochastic Partial Differential Equations

TL;DR

This paper introduces anisotropic stochastic PDEs for score-based generative modeling, which preserve geometric data structures longer, leading to improved image generation quality and fidelity.

Contribution

It proposes a novel anisotropic SPDE framework that maintains geometric structure during diffusion, enhancing the quality of generated images over traditional isotropic models.

Findings

Anisotropic diffusion achieves superior image quality metrics.

Consistent improvements over baseline SDE models.

Effective in both unconditional and conditional image generation tasks.

Abstract

Score-based generative modeling (SBGM) has achieved state-of-the-art performance in image generation, with the quality of generated images being highly dependent on the design of the forward (diffusion) process. Among these, models based on stochastic differential equations (SDEs) have proven particularly effective. While traditional methods aim to progressively destroy all image information to enable reconstruction from pure noise, we propose a class of anisotropic stochastic partial differential equations (SPDEs) that preserve the geometric structure of the data over longer time scales throughout the transformation. These SPDEs consist of a drift term that enforces deterministic destruction via structured smoothing, and a diffusion coefficient that enables random destruction through noise injection. Both components are governed by anisotropy coefficients, enabling controlled, direction-dependent information degradation. This framework provides the theoretical foundation for a novel anisotropic score-based generative model. By retaining geometric structure for longer time scales, the backward generative process can exploit residual geometric cues, leading to improved reconstruction fidelity. We empirically validate this improvement in a proof-of-concept implementation on unconditional image generation, showing that anisotropic diffusion can achieve superior image quality metrics. We demonstrate consistent improvements in both pixel and latent space experiments over the SDE-driven baseline as well as over the state-of-the-art Flow Matching approach. Finally, we demonstrate the effectiveness of the introduced anisotropy in a conditional stroke-to-image generation task.