Provable Accuracy Bounds for Hybrid Dynamical Optimization and Sampling

TL;DR

This paper establishes non-asymptotic convergence guarantees for hybrid analog/digital local search algorithms in dynamical optimization and sampling, linking device variation and hyperparameters to performance.

Contribution

It introduces a theoretical framework that provides convergence bounds for hybrid LNLS algorithms by reducing them to block Langevin Diffusion models.

Findings

Proves exponential KL-divergence convergence for ideal DXs.

Provides explicit bounds on 2-Wasserstein bias with device variation.

Links device variation, hyperparameters, and performance through a closed-form expression.

Abstract

Analog dynamical accelerators (DXs) are a growing sub-field in computer architecture research, offering order-of-magnitude gains in power efficiency and latency over traditional digital methods in several machine learning, optimization, and sampling tasks. However, limited-capacity accelerators require hybrid analog/digital algorithms to solve real-world problems, commonly using large-neighborhood local search (LNLS) frameworks. Unlike fully digital algorithms, hybrid LNLS has no non-asymptotic convergence guarantees and no principled hyperparameter selection schemes, particularly limiting cross-device training and inference. In this work, we provide non-asymptotic convergence guarantees for hybrid LNLS by reducing to block Langevin Diffusion (BLD) algorithms. Adapting tools from classical sampling theory, we prove exponential KL-divergence convergence for randomized and cyclic block selection strategies using ideal DXs. With finite device variation, we provide explicit bounds on the 2-Wasserstein bias in terms of step duration, noise strength, and function parameters. Our BLD model provides a key link between established theory and novel computing platforms, and our theoretical results provide a closed-form expression linking device variation, algorithm hyperparameters, and performance.