Parallelized Multi-Agent Bayesian Optimization in Lava

TL;DR

This paper presents Lava Multi-Agent Optimization (LMAO), a framework enabling distributed Bayesian optimization in neuromorphic systems, improving scalability and efficiency in complex parameter search tasks.

Contribution

LMAO introduces a novel distributed optimization framework with support for various search methods and mathematical precisions within the Lava neuromorphic software ecosystem.

Findings

LMAO scales efficiently with multiple processes.

Reduces cumulative runtime in optimization tasks.

Minimizes convergence to local optima.

Abstract

In parallel with the continuously increasing parameter space dimensionality, search and optimization algorithms should support distributed parameter evaluations to reduce cumulative runtime. Intel's neuromorphic optimization library, Lava-Optimization, was introduced as an abstract optimization system compatible with neuromorphic systems developed in the broader Lava software framework. In this work, we introduce Lava Multi-Agent Optimization (LMAO) with native support for distributed parameter evaluations communicating with a central Bayesian optimization system. LMAO provides an abstract framework for deploying distributed optimization and search algorithms within the Lava software framework. Moreover, LMAO introduces support for random and grid search along with process connections across multiple levels of mathematical precision. We evaluate the algorithmic performance of LMAO with a traditional non-convex optimization problem, a fixed-precision transductive spiking graph neural network for citation graph classification, and a neuromorphic satellite scheduling problem. Our results highlight LMAO's efficient scaling to multiple processes, reducing cumulative runtime and minimizing the likelihood of converging to local optima.