PG-ControlNet: A Physics-Guided ControlNet for Generative Spatially Varying Image Deblurring

TL;DR

This paper introduces PG-ControlNet, a novel physics-guided generative model for spatially varying image deblurring that combines physical constraints with a powerful generative prior, achieving superior results in complex blur scenarios.

Contribution

It proposes modeling the degradation as a dense continuum of kernels and integrating this with ControlNet to improve deblurring accuracy and perceptual quality.

Findings

Outperforms state-of-the-art methods in severely blurred scenarios

Effectively balances physical accuracy and perceptual realism

Captures minute variations in degradation patterns

Abstract

Spatially varying image deblurring remains a fundamentally ill-posed problem, especially when degradations arise from complex mixtures of motion and other forms of blur under significant noise. State-of-the-art learning-based approaches generally fall into two paradigms: model-based deep unrolling methods that enforce physical constraints by modeling the degradations, but often produce over-smoothed, artifact-laden textures, and generative models that achieve superior perceptual quality yet hallucinate details due to weak physical constraints. In this paper, we propose a novel framework that uniquely reconciles these paradigms by taming a powerful generative prior with explicit, dense physical constraints. Rather than oversimplifying the degradation field, we model it as a dense continuum of high-dimensional compressed kernels, ensuring that minute variations in motion and other degradation patterns are captured. We leverage this rich descriptor field to condition a ControlNet architecture, strongly guiding the diffusion sampling process. Extensive experiments demonstrate that our method effectively bridges the gap between physical accuracy and perceptual realism, outperforming state-of-the-art model-based methods as well as generative baselines in challenging, severely blurred scenarios.