Deep Spectral Prior

TL;DR

Deep Spectral Prior (DSP) is an unsupervised image reconstruction framework operating in the frequency domain, which self-regularizes and outperforms previous methods like DIP by leveraging spectral mode separation and implicit manifold projection.

Contribution

DSP introduces a novel spectral domain approach with a theoretical analysis, eliminating the need for early stopping and explicit priors in unsupervised image reconstruction.

Findings

DSP outperforms DIP and baselines in denoising, inpainting, and deblurring.

Theoretical analysis shows frequency-dependent descent dynamics.

DSP achieves stable, high-fidelity reconstructions without supervision.

Abstract

We introduce the Deep Spectral Prior (DSP), a new framework for unsupervised image reconstruction that operates entirely in the complex frequency domain. Unlike the Deep Image Prior (DIP), which optimises pixel-level errors and is highly sensitive to overfitting, DSP performs joint learning of amplitude and phase to capture the full spectral structure of images. We derive a rigorous theoretical characterisation of DSP's optimisation dynamics, proving that it follows frequency-dependent descent trajectories that separate informative low-frequency modes from stochastic high-frequency noise. This spectral mode separation explains DSP's self-regularising behaviour and, for the first time, formally establishes the elimination of DIP's major limitation-its reliance on manual early stopping. Moreover, DSP induces an implicit projection onto a frequency-consistent manifold, ensuring convergence to stable, physically plausible reconstructions without explicit priors or supervision. Extensive experiments on denoising, inpainting, and deblurring demonstrate that DSP consistently surpasses DIP and other unsupervised baselines, achieving superior fidelity, robustness, and theoretical interpretability within a unified, unsupervised data-free framework.