Real-Time Black-Box Optimization for Dynamic Discrete Environments Using Embedded Ising Machines

TL;DR

This paper introduces a heuristic multi-armed bandit approach utilizing Ising machines to efficiently optimize discrete actions in real-time dynamic environments, demonstrated in wireless communication systems.

Contribution

It extends static environment BBO methods with Ising machines to handle dynamic, discrete optimization problems in real-time systems.

Findings

Effective exploration of actions considering variable interactions.

Adaptability demonstrated in wireless communication scenarios.

Outperforms traditional methods in dynamic settings.

Abstract

Many real-time systems require the optimization of discrete variables. Black-box optimization (BBO) algorithms and multi-armed bandit (MAB) algorithms perform optimization by repeatedly taking actions and observing the corresponding instant rewards without any prior knowledge. Recently, a BBO method using an Ising machine has been proposed to find the best action that is represented by a combination of discrete values and maximizes the instant reward in static environments. In contrast, dynamic environments, where real-time systems operate, necessitate MAB algorithms that maximize the average reward over multiple trials. However, due to the enormous number of actions resulting from the combinatorial nature of discrete optimization, conventional MAB algorithms cannot effectively optimize dynamic, discrete environments. Here, we show a heuristic MAB method for dynamic, discrete environments by extending the BBO method, in which an Ising machine effectively explores the actions while considering interactions between variables and changes in dynamic environments. We demonstrate the dynamic adaptability of the proposed method in a wireless communication system with moving users.