Optimization of Graph Total Variation via Active-Set-based Combinatorial Reconditioning

TL;DR

This paper introduces an adaptive preconditioning strategy for structured convex optimization on graphs, improving convergence rates by analyzing active sets and employing nested-forest decompositions, with practical heuristics validated through experiments.

Contribution

It proposes a novel active-set-based preconditioning method for proximal algorithms on graph-structured convex problems, enhancing convergence efficiency.

Findings

Nested-forest decomposition guarantees local linear convergence.

The proposed heuristic effectively implements the nested decompositions.

Experiments show competitive performance of the reconditioning strategy.

Abstract

Structured convex optimization on weighted graphs finds numerous applications in machine learning and computer vision. In this work, we propose a novel adaptive preconditioning strategy for proximal algorithms on this problem class. Our preconditioner is driven by a sharp analysis of the local linear convergence rate depending on the "active set" at the current iterate. We show that nested-forest decomposition of the inactive edges yields a guaranteed local linear convergence rate. Further, we propose a practical greedy heuristic which realizes such nested decompositions and show in several numerical experiments that our reconditioning strategy, when applied to proximal gradient or primal-dual hybrid gradient algorithm, achieves competitive performances. Our results suggest that local convergence analysis can serve as a guideline for selecting variable metrics in proximal algorithms.