Combinatorial Preconditioners for Proximal Algorithms on Graphs

TL;DR

This paper introduces a new graph-based preconditioning technique for proximal algorithms, utilizing graph partitions into forests to improve convergence and efficiency in large-scale machine learning and vision tasks.

Contribution

It proposes a combinatorial preconditioning method based on graph decompositions, with theoretical guarantees and practical greedy algorithms for large-scale problems.

Findings

Achieves competitive performance in machine learning applications

Provides theoretical bounds on condition numbers

Develops efficient greedy algorithms for graph decompositions

Abstract

We present a novel preconditioning technique for proximal optimization methods that relies on graph algorithms to construct effective preconditioners. Such combinatorial preconditioners arise from partitioning the graph into forests. We prove that certain decompositions lead to a theoretically optimal condition number. We also show how ideal decompositions can be realized using matroid partitioning and propose efficient greedy variants thereof for large-scale problems. Coupled with specialized solvers for the resulting scaled proximal subproblems, the preconditioned algorithm achieves competitive performance in machine learning and vision applications.