A Theory of Composition and Duality of Extremal Optimal Fixed-Point Algorithms

TL;DR

This paper uncovers a combinatorial structure for optimal fixed-point algorithms, enabling systematic design and analysis through diagrammatic composition and duality, leading to new robust algorithms.

Contribution

It introduces a diagrammatic framework for understanding and constructing extremal optimal fixed-point algorithms, including duality and composition principles.

Findings

Characterizes the set of extremal optimal algorithms as vertices of a factorially large polytope.

Develops a diagrammatic method to compose and analyze optimal algorithms.

Proposes new algorithms with quasi-anytime guarantees and robustness to nonexpansiveness violations.

Abstract

In this work, we reveal a rich combinatorial structure underlying exact minimax optimal algorithms for classical nonexpansive fixed-point problems. This viewpoint unifies all extremal optimal methods and provides a systematic and practical framework for designing new algorithms via diagrams. Specifically, we study fixed-step algorithms represented by a lower triangular matrix H, and show that the set of optimal (N-1)-step algorithms has exactly (N-1)! vertices (extremal algorithms), each of which naturally corresponds to an arc diagram, a graph that encodes its convergence proof. Using these arc diagrams, we can compose, decompose, and analyze the properties of distinct optimal vertex algorithms. Furthermore, we determine when the H-dual operation, given by taking the anti-diagonal transpose of H, preserves the optimality of a vertex algorithm, and in such cases we characterize the convergence proof of the dual algorithm. Based on this machinery, we develop new optimal algorithms with quasi-anytime guarantees; that is, they admit an increasing integer sequence such that the corresponding iterates have the optimal residual guarantees, and are additionally robust to fixed-point operators that violate nonexpansiveness.