Online Architecture Search for Compressed Sensing based on Hypergradient Descent

TL;DR

This paper introduces an online hypergradient descent method for architecture search in compressed sensing algorithms, improving performance without retraining when environmental conditions change.

Contribution

It proposes HGD-AS-ISTA and HGD-AS-FISTA, novel algorithms that use hypergradient descent for online architecture search, eliminating retraining overhead.

Findings

Enhanced performance over traditional ISTA/FISTA

No need for re-training when environment changes

Effective online hyperparameter optimization

Abstract

AS-ISTA (Architecture Searched-Iterative Shrinkage Thresholding Algorithm) and AS-FISTA (AS-Fast ISTA) are compressed sensing algorithms introducing structural parameters to ISTA and FISTA to enable architecture search within the iterative process. The structural parameters are determined using deep unfolding, but this approach requires training data and the large overhead of training time. In this paper, we propose HGD-AS-ISTA (Hypergradient Descent-AS-ISTA) and HGD-AS-FISTA that use hypergradient descent, which is an online hyperparameter optimization method, to determine the structural parameters. Experimental results show that the proposed method improves performance of the conventional ISTA/FISTA while avoiding the need for re-training when the environment changes.