On the filtering effect of iterative regularization algorithms for linear least-squares problems

TL;DR

This paper analyzes how iterative regularization algorithms for linear least-squares problems filter singular values, emphasizing the roles of scaling matrices and non-negative constraints in improving solution stability and accuracy.

Contribution

It provides a detailed analysis of classical and recent gradient methods, highlighting the impact of scaling and constraints on the filtering effect in regularization.

Findings

Scaling matrices influence the filtering of singular values.

Non-negative constraints affect the stability of the solution.

The analysis clarifies the role of regularization in iterative algorithms.

Abstract

Many real-world applications are addressed through a linear least-squares problem formulation, whose solution is calculated by means of an iterative approach. A huge amount of studies has been carried out in the optimization field to provide the fastest methods for the reconstruction of the solution, involving choices of adaptive parameters and scaling matrices. However, in presence of an ill-conditioned model and real data, the need of a regularized solution instead of the least-squares one changed the point of view in favour of iterative algorithms able to combine a fast execution with a stable behaviour with respect to the restoration error. In this paper we want to analyze some classical and recent gradient approaches for the linear least-squares problem by looking at their way of filtering the singular values, showing in particular the effects of scaling matrices and non-negative constraints in recovering the correct filters of the solution.