The Combinatorial Multi-Bandit Problem and its Application to Energy Management

TL;DR

This paper investigates a combinatorial multi-bandit problem motivated by energy management, proposing algorithms that leverage observability for improved exploration, and demonstrates their effectiveness in a large-scale energy application.

Contribution

It introduces a generalized lower regret bound for combinatorial multi-bandits with observable outcomes and develops algorithms combining exploration with mathematical programming.

Findings

Effective learning of action assignments for 150 bandits within 365 episodes

Parallelized exploration improves regret bounds in combinatorial bandits

Algorithms outperform baseline methods in energy management simulations

Abstract

We study a Combinatorial Multi-Bandit Problem motivated by applications in energy systems management. Given multiple probabilistic multi-arm bandits with unknown outcome distributions, the task is to optimize the value of a combinatorial objective function mapping the vector of individual bandit outcomes to a single scalar reward. Unlike in single-bandit problems with multi-dimensional action space, the outcomes of the individual bandits are observable in our setting and the objective function is known. Guided by the hypothesis that individual observability enables better trade-offs between exploration and exploitation, we generalize the lower regret bound for single bandits, showing that indeed for multiple bandits it admits parallelized exploration. For our energy management application we propose a range of algorithms that combine exploration principles for multi-arm bandits with mathematical programming. In an experimental study we demonstrate the effectiveness of our approach to learn action assignments for 150 bandits, each having 24 actions, within a horizon of 365 episodes.