Differentiable SVD based on Moore-Penrose Pseudoinverse for Inverse Imaging Problems

TL;DR

This paper introduces a differentiable SVD method based on the Moore-Penrose pseudoinverse to improve numerical stability in inverse imaging problems, enabling more reliable deep learning models.

Contribution

It is the first to analyze the non-differentiability of SVD and proposes a novel differentiable SVD solution using the Moore-Penrose pseudoinverse.

Findings

Enhanced numerical stability in inverse imaging tasks

Effective in color image compressed sensing

Improved dynamic MRI reconstruction results

Abstract

Low-rank regularization-based deep unrolling networks have achieved remarkable success in various inverse imaging problems (IIPs). However, the singular value decomposition (SVD) is non-differentiable when duplicated singular values occur, leading to severe numerical instability during training. In this paper, we propose a differentiable SVD based on the Moore-Penrose pseudoinverse to address this issue. To the best of our knowledge, this is the first work to provide a comprehensive analysis of the differentiability of the trivial SVD. Specifically, we show that the non-differentiability of SVD is essentially due to an underdetermined system of linear equations arising in the derivation process. We utilize the Moore-Penrose pseudoinverse to solve the system, thereby proposing a differentiable SVD. A numerical stability analysis in the context of IIPs is provided. Experimental results in color image compressed sensing and dynamic MRI reconstruction show that our proposed differentiable SVD can effectively address the numerical instability issue while ensuring computational precision. Code is available at https://github.com/yhao-z/SVD-inv.