i-DEQ: A stable inertial deep equilibrium model for image restoration

TL;DR

The paper introduces i-DEQ, a stable and efficient inertial deep equilibrium model for image restoration that improves training stability and reduces inference time while maintaining high reconstruction quality.

Contribution

It proposes an inertial DEQ with momentum for better convergence, stability, and speed in image restoration tasks, addressing limitations of traditional DEQs.

Findings

i-DEQ achieves comparable reconstruction quality to state-of-the-art methods.

Training stability and robustness are significantly improved with i-DEQ.

Inference time is reduced by a factor of two compared to standard DEQs.

Abstract

Deep Equilibrium Models (DEQs) are an established framework for image restoration that learn a problem-adapted regularization by solving a fixed-point (i.e. equilibrium) problem. While flexible and expressive, DEQs are often hindered by high computational cost and training instability. We propose an inertial DEQ (i-DEQ) that learns an explicit nonconvex regularization within the DEQ formulation. By using momentum within the fixed-point iterations, i-DEQ has convergence guarantees and accelerated rates. Moreover, we observe that i-DEQ is significantly more stable during the training and robust to rough initialization than DEQs. Numerical experiments on various linear and nonlinear inverse problems demonstrate that i-DEQ achieves reconstruction quality comparable to state-of-the-art methods, while reducing DEQ's inference time by a factor of two.