Probabilistic Verification of Approximate Algorithms with Unstructured Errors: Application to Fully Inexact Generalized ADMM

TL;DR

This paper analyzes the convergence of an approximate ADMM algorithm with unstructured errors, providing probabilistic bounds and validating results through simulations and practical applications in regression problems.

Contribution

It introduces probabilistic convergence bounds for inexact ADMM with unstructured errors, extending classical results to more realistic computational noise models.

Findings

Probabilistic bounds accurately predict convergence behavior.

Validation on synthetic and real regression problems.

Adaptive bounds outperform classical deterministic bounds.

Abstract

We analyse the convergence of an approximate, fully inexact, ADMM algorithm under additive, deterministic and probabilistic error models. We consider the generalized ADMM scheme that is derived from generalized Lagrangian penalty with additive (smoothing) adaptive-metric quadratic proximal perturbations. We derive explicit deterministic and probabilistic convergence upper bounds for the lower-C2 nonconvex case as well as the convex case under the Lipschitz continuity condition. We also present more practical conditions on the proximal errors under which convergence of the approximate ADMM to a suboptimal solution is guaranteed with high probability. We consider statistically and dynamically-unstructured conditional mean independent bounded error sequences. We validate our results using both simulated and practical software and algorithmic computational perturbations. We apply the proposed algorithm to a synthetic LASSO and robust regression with k-support norm regularization problems and test our proposed bounds under different computational noise levels. Compared to classical convergence results, the adaptive probabilistic bounds are more accurate in predicting the distance from the optimal set and parasitic residual error under different sources of inaccuracies.