Faster gradient descent and the efficient recovery of images

TL;DR

This paper investigates faster gradient descent algorithms for image deblurring and denoising, demonstrating their advantages over traditional methods in ill-posed inverse problems, and proposes efficient schemes for practical implementation.

Contribution

The paper introduces and analyzes faster gradient descent methods tailored for image restoration tasks, including schemes that are independent of step size and effective with noisy and blurred images.

Findings

Faster gradient descent methods converge more quickly in image deblurring and denoising.

Proposed schemes are highly efficient and step size independent.

Faster methods outperform steepest descent in ill-posed inverse problems.

Abstract

Much recent attention has been devoted to gradient descent algorithms where the steepest descent step size is replaced by a similar one from a previous iteration or gets updated only once every second step, thus forming a {\em faster gradient descent method}. For unconstrained convex quadratic optimization these methods can converge much faster than steepest descent. But the context of interest here is application to certain ill-posed inverse problems, where the steepest descent method is known to have a smoothing, regularizing effect, and where a strict optimization solution is not necessary. Specifically, in this paper we examine the effect of replacing steepest descent by a faster gradient descent algorithm in the practical context of image deblurring and denoising tasks. We also propose several highly efficient schemes for carrying out these tasks independently of the step size selection, as well as a scheme for the case where both blur and significant noise are present. In the above context there are situations where many steepest descent steps are required, thus building slowness into the solution procedure. Our general conclusion regarding gradient descent methods is that in such cases the faster gradient descent methods offer substantial advantages. In other situations where no such slowness buildup arises the steepest descent method can still be very effective.