Dynamic Sample Complexity for Exact Sparse Recovery using Sequential Iterative Hard Thresholding

TL;DR

This paper introduces a sequential version of the iterative hard thresholding algorithm for sparse recovery, demonstrating that it achieves rapid error decay with fewer measurements and improved recovery probability in a dynamic measurement setting.

Contribution

The paper proposes the sequential IHT algorithm and provides theoretical analysis showing its effectiveness in dynamic measurement scenarios with stochastic matrices.

Findings

Error decay is rapid under certain sample complexity bounds.

Small, sporadic measurements do not significantly hinder error decay.

Numerical experiments show improved recovery probability over offline IHT.

Abstract

In this paper we consider the problem of exact recovery of a fixed sparse vector with the measurement matrices sequentially arriving along with corresponding measurements. We propose an extension of the iterative hard thresholding (IHT) algorithm, termed as sequential IHT (SIHT) which breaks the total time horizon into several phases such that IHT is executed in each of these phases using a fixed measurement matrix obtained at the beginning of that phase. We consider a stochastic setting where the measurement matrices obtained at each phase are independent samples of a sub Gaussian random matrix. We prove that if a certain dynamic sample complexity that depends on the sizes of the measurement matrices at each phase, along with their duration and the number of phases, satisfy certain lower bound, the estimation error of SIHT over a fixed time horizon decays rapidly. Interestingly, this bound reveals that the probability of decay of estimation error is hardly affected even if very small number measurements are sporadically used in different phases. This theoretical observation is also corroborated using numerical experiments demonstrating that SIHT enjoys improved probability of recovery compared to offline IHT.