On the convergence of the IRLS algorithm in Non-Local Patch Regression

TL;DR

This paper analyzes the convergence properties of the IRLS algorithm used in Non-Local Patch Regression for image denoising, demonstrating global convergence in convex cases and local convergence in non-convex cases, explained via a majorize-minimize framework.

Contribution

It provides a theoretical explanation for the IRLS algorithm's convergence behavior in NLPR, extending understanding to both convex and non-convex regimes.

Findings

IRLS is globally convergent for 1 ≤ p ≤ 2.

IRLS is locally convergent for 0 < p < 1.

Majorize-minimize framework explains convergence observations.

Abstract

Recently, it was demonstrated in [CS2012,CS2013] that the robustness of the classical Non-Local Means (NLM) algorithm [BCM2005] can be improved by incorporating $\ell^p (0 < p \leq 2)$ regression into the NLM framework. This general optimization framework, called Non-Local Patch Regression (NLPR), contains NLM as a special case. Denoising results on synthetic and natural images show that NLPR consistently performs better than NLM beyond a moderate noise level, and significantly so when $p$ is close to zero. An iteratively reweighted least-squares (IRLS) algorithm was proposed for solving the regression problem in NLPR, where the NLM output was used to initialize the iterations. Based on exhaustive numerical experiments, we observe that the IRLS algorithm is globally convergent (for arbitrary initialization) in the convex regime $1 \leq p \leq 2$, and locally convergent (fails very rarely using NLM initialization) in the non-convex regime $0 < p < 1$. In this letter, we adapt the "majorize-minimize" framework introduced in [Voss1980] to explain these observations. [CS2012] Chaudhury et al. (2012), "Non-local Euclidean medians," IEEE Signal Processing Letters. [CS2013] Chaudhury et al. (2013), "Non-local patch regression: Robust image denoising in patch space," IEEE ICASSP. [BCM2005] Buades et al. (2005), "A review of image denoising algorithms, with a new one," Multiscale Modeling and Simulation. [Voss1980] Voss et al. (1980), "Linear convergence of generalized Weiszfeld's method," Computing.