Integrating Multi-Armed Bandit, Active Learning, and Distributed Computing for Scalable Optimization

TL;DR

This paper introduces ALMAB-DC, a scalable, modular framework combining active learning, multi-armed bandits, and distributed computing to efficiently solve complex black-box optimization problems with theoretical guarantees and superior empirical performance.

Contribution

The paper presents ALMAB-DC, a novel unified framework that integrates multiple techniques for scalable black-box optimization, with theoretical analysis and extensive empirical validation.

Findings

Outperforms state-of-the-art black-box optimizers on benchmarks.

Provides theoretical regret bounds for UCB and Thompson sampling variants.

Demonstrates scalability and efficiency in high-dimensional, resource-intensive tasks.

Abstract

Modern optimization problems in scientific and engineering domains often rely on expensive black-box evaluations, such as those arising in physical simulations or deep learning pipelines, where gradient information is unavailable or unreliable. In these settings, conventional optimization methods quickly become impractical due to prohibitive computational costs and poor scalability. We propose ALMAB-DC, a unified and modular framework for scalable black-box optimization that integrates active learning, multi-armed bandits, and distributed computing, with optional GPU acceleration. The framework leverages surrogate modeling and information-theoretic acquisition functions to guide informative sample selection, while bandit-based controllers dynamically allocate computational resources across candidate evaluations in a statistically principled manner. These decisions are executed asynchronously within a distributed multi-agent system, enabling high-throughput parallel evaluation. We establish theoretical regret bounds for both UCB-based and Thompson-sampling-based variants and develop a scalability analysis grounded in Amdahl's and Gustafson's laws. Empirical results across synthetic benchmarks, reinforcement learning tasks, and scientific simulation problems demonstrate that ALMAB-DC consistently outperforms state-of-the-art black-box optimizers. By design, ALMAB-DC is modular, uncertainty-aware, and extensible, making it particularly well suited for high-dimensional, resource-intensive optimization challenges.