Measuring Robustness in Deep Learning Based Compressive Sensing

TL;DR

This paper evaluates the robustness of deep learning methods for image reconstruction in compressive sensing, revealing vulnerabilities to adversarial perturbations and distribution shifts, while also showing that higher reconstruction quality correlates with better detail recovery.

Contribution

The study provides a comprehensive measurement of robustness in deep learning-based image reconstruction, comparing trained, un-trained, and traditional methods under adversarial and distribution shift scenarios.

Findings

Both trained and un-trained methods are vulnerable to adversarial perturbations.

Methods tuned for specific datasets suffer similarly from distribution shifts.

Higher reconstruction quality correlates with better recovery of fine details.

Abstract

Deep neural networks give state-of-the-art accuracy for reconstructing images from few and noisy measurements, a problem arising for example in accelerated magnetic resonance imaging (MRI). However, recent works have raised concerns that deep-learning-based image reconstruction methods are sensitive to perturbations and are less robust than traditional methods: Neural networks (i) may be sensitive to small, yet adversarially-selected perturbations, (ii) may perform poorly under distribution shifts, and (iii) may fail to recover small but important features in an image. In order to understand the sensitivity to such perturbations, in this work, we measure the robustness of different approaches for image reconstruction including trained and un-trained neural networks as well as traditional sparsity-based methods. We find, contrary to prior works, that both trained and un-trained methods are vulnerable to adversarial perturbations. Moreover, both trained and un-trained methods tuned for a particular dataset suffer very similarly from distribution shifts. Finally, we demonstrate that an image reconstruction method that achieves higher reconstruction quality, also performs better in terms of accurately recovering fine details. Our results indicate that the state-of-the-art deep-learning-based image reconstruction methods provide improved performance than traditional methods without compromising robustness.