DIPA: Distilled Preconditioned Algorithms for Solving Imaging Inverse Problems

TL;DR

This paper introduces DIPA, a novel approach that uses teacher-guided distillation to optimize preconditioning operators, enhancing reconstruction quality in imaging inverse problems across multiple modalities.

Contribution

The paper proposes DIPA, a method that learns preconditioning operators through distillation, improving convergence and reconstruction quality in ill-conditioned imaging inverse problems.

Findings

DIPA improves reconstruction quality in MRI, compressed sensing, and super-resolution imaging.

Both linear and non-linear preconditioning operators are effective, with non-linear offering better scalability.

Distillation-based preconditioning outperforms traditional methods in various imaging tasks.

Abstract

Solving imaging inverse problems has usually been addressed by designing proper prior models of the underlying signal. However, minimizing the data fidelity term poses significant challenges due to the ill-conditioned sensing matrix caused by physical constraints in the acquisition system. Thus, preconditioning techniques have been adopted in classical optimization theory to address ill-conditioned data-fidelity minimization by transforming the algorithm gradient step to achieve faster convergence and better numerical stability. We extend the preconditioning concept beyond convergence acceleration and use it to improve reconstruction quality. We introduce DIPA: Distilled Preconditioned Algorithms, where a preconditioning operator (PO) is optimized using teacher-guided distillation criteria. Unlike standard model-compression KD, the teacher and student differ by the sensing operators available during reconstruction: the teacher uses a simulated, better-conditioned, and more informative sensing matrix, whereas the student uses the physically feasible sensing matrix. We design different distillation loss functions to transfer different properties of the teacher algorithm to the preconditioned student. The PO can be linear (L-DIPA), allowing interpretability, or non-linear (N-DIPA), parametrized by a neural network, offering better scalability. We validate the proposed PO design across several imaging modalities, including magnetic resonance imaging, compressed sensing, and super-resolution imaging.