Dynamic Thresholding Algorithm with Memory for Linear Inverse Problems

TL;DR

The paper introduces DTAM, a new dynamic thresholding algorithm with memory that efficiently solves large-scale linear inverse problems by reducing computational complexity while maintaining high accuracy.

Contribution

It proposes DTAM, a novel iterative method combining memory and sparse search directions, significantly lowering computational costs compared to existing ROTP algorithms.

Findings

DTAM achieves comparable accuracy to mainstream algorithms.

It significantly outperforms ROTP in speed, especially for low sparsity signals.

The algorithm effectively solves problems with matrices satisfying the restricted isometry property.

Abstract

The relaxed optimal $k$-thresholding pursuit (ROTP) is a recent algorithm for linear inverse problems. This algorithm is based on the optimal $k$-thresholding technique which performs vector thresholding and error metric reduction simultaneously. Although ROTP can be used to solve small to medium-sized linear inverse problems, the computational cost of this algorithm is high when solving large-scale problems. By merging the optimal $k$-thresholding technique and iterative method with memory as well as optimization with sparse search directions, we propose the so-called dynamic thresholding algorithm with memory (DTAM), which iteratively and dynamically selects vector bases to construct the problem solution. At every step, the algorithm uses more than one or all iterates generated so far to construct a new search direction, and solves only the small-sized quadratic subproblems at every iteration. Thus the computational complexity of DTAM is remarkably lower than that of ROTP-type methods. It turns out that DTAM can locate the solution of linear inverse problems if the matrix involved satisfies the restricted isometry property. Experiments on synthetic data, audio signal reconstruction and image denoising demonstrate that the proposed algorithm performs comparably to several mainstream thresholding and greedy algorithms, and it works much faster than the ROTP-type algorithms especially when the sparsity level of signal is relatively low.