How to warm-start your unfolding network

TL;DR

This paper introduces C-DEC, an ensemble framework that enhances overparameterized unfolding networks for compressed sensing by using continuation techniques and a specialized loss function, leading to better performance and smoother optimization landscapes.

Contribution

The paper proposes a novel ensemble approach combining continuation with overparameterized unfolding networks and a log-cosh loss for improved compressed sensing reconstruction.

Findings

C-DEC achieves smoother loss landscapes.

C-DEC improves reconstruction accuracy.

C-DEC generalizes well across datasets.

Abstract

We present a new ensemble framework for boosting the performance of overparameterized unfolding networks solving the compressed sensing problem. We combine a state-of-the-art overparameterized unfolding network with a continuation technique, to warm-start a crucial quantity of the said network's architecture; we coin the resulting continued network C-DEC. Moreover, for training and evaluating C-DEC, we incorporate the log-cosh loss function, which enjoys both linear and quadratic behavior. Finally, we numerically assess C-DEC's performance on real-world images. Results showcase that the combination of continuation with the overparameterized unfolded architecture, trained and evaluated with the chosen loss function, yields smoother loss landscapes and improved reconstruction and generalization performance of C-DEC, consistently for all datasets.