Implicit Regularization Effects of the Sobolev Norms in Image Processing

TL;DR

This paper explores how Sobolev norms inherently regularize image processing tasks by weighting frequency components, providing insights into their effects on optimization and noise handling.

Contribution

It analyzes the implicit regularization effects of Sobolev norms in image processing and connects these effects to Bayesian perspectives and optimization dynamics.

Findings

Sobolev norms impose frequency-dependent weights that act as implicit regularizers.

Using Sobolev norms as data-fitting terms influences the convergence of gradient-based methods.

Numerical experiments confirm the regularization effects in various image processing tasks.

Abstract

In this paper, we propose to use the general $L^2$-based Sobolev norms, i.e., $H^s$ norms where $s\in \mathbb{R}$, to measure the data discrepancy due to noise in image processing tasks that are formulated as optimization problems. As opposed to a popular trend of developing regularization methods, we emphasize that an implicit regularization effect can be achieved through the class of Sobolev norms as the data-fitting term. Specifically, we analyze that the implicit regularization comes from the weights that the $H^s$ norm imposes on different frequency contents of an underlying image. We further analyze the underlying noise assumption of using the Sobolev norm as the data-fitting term from a Bayesian perspective, build the connections with the Sobolev gradient-based methods and discuss the preconditioning effects on the convergence rate of the gradient descent algorithm, leading to a better understanding of functional spaces/metrics and the optimization process involved in image processing. Numerical results in full waveform inversion, image denoising and deblurring demonstrate the implicit regularization effects.