Variable-Wise Diagonal Preconditioning for Primal-Dual Splitting: Design and Applications

TL;DR

This paper introduces a novel operator norm-based variable-wise diagonal preconditioning method for primal-dual splitting algorithms, improving convergence speed and applicability to large-scale problems without requiring explicit matrix representations.

Contribution

It proposes OVDP, a preconditioning design that uses operator norms and variable-wise approach, overcoming limitations of previous methods in handling linear operators and proximity operators.

Findings

Effective in hyperspectral image noise removal

Improves convergence speed of primal-dual algorithms

Applicable to hyperspectral unmixing and graph signal recovery

Abstract

This paper proposes a method for designing diagonal preconditioners for a preconditioned primal-dual splitting method (P-PDS), an efficient algorithm that solves nonsmooth convex optimization problems. To speed up the convergence of P-PDS, a design method has been proposed to automatically determine appropriate preconditioners from the problem structure. However, the existing method has two limitations. One is that it directly accesses all elements of matrices representing linear operators involved in a given problem, which is inconvenient for handling linear operators implemented as procedures rather than matrices. The other is that it takes an element-wise preconditioning approach, which turns certain types of proximity operators into analytically intractable forms. To overcome these limitations, we establish an Operator norm-based design method of Variable-wise Diagonal Preconditioning (OVDP). First, OVDP constructs diagonal preconditioners using only (upper bounds) of the operator norms of linear operators, thus eliminating the need for their explicit matrix representations. Furthermore, since OVDP takes a variable-wise preconditioning approach, it keeps any proximity operator analytically computable. We also prove that our preconditioners satisfy the convergence condition of P-PDS. Finally, we demonstrate the effectiveness and usefulness of OVDP through applications to mixed noise removal of hyperspectral images, hyperspectral unmixing, and graph signal recovery.