Principled Design of Diffusion-based Optimizers for Inverse Problems

TL;DR

This paper introduces principled reparameterizations and the OptDiff pipeline for diffusion-based inverse problem solvers, enabling hyperparameter reusability and faster, higher-quality reconstructions.

Contribution

It proposes invariance-inducing reparameterizations and an optimization-based framework to improve hyperparameter tuning and inference speed in diffusion models for inverse problems.

Findings

Substantial speedups in image reconstruction tasks

Improved image quality in deblurring and super-resolution

Reusability of hyperparameters across multiple problems

Abstract

Score-based diffusion models achieve state-of-the-art performance for inverse problems, but their practical deployment is hindered by long inference times and cumbersome hyperparameter tuning. While pretrained diffusion models can be reused across tasks without retraining, inference-time hyperparameters such as the noise schedule and posterior sampling weights typically require ad-hoc adjustment for each problem setup. We propose principled reparameterizations that induce invariances, allowing the same hyperparameters to be reused across multiple problems without re-tuning. In addition, building on the RED-diff framework, which reformulates posterior sampling as an optimization problem, we further develop the OptDiff pipeline. OptDiff provides a simplified tuning framework that facilitates the integration of convex optimization tools to accelerate inference. Experiments on image reconstruction, deblurring, and super-resolution show substantial speedups and improved image quality.