Solving multi-armed bandit problems using a chaotic microresonator comb

TL;DR

This paper introduces a novel photonic approach using a chaotic microresonator frequency comb to efficiently solve multi-armed bandit problems, demonstrating high scalability and competitive performance with existing algorithms.

Contribution

The paper presents the first use of a chaos comb in solving MAB problems, achieving high scalability and performance comparable to traditional and photonic algorithms.

Findings

Successfully addressed 44-slot machine MAB with chaos comb

Achieved power-law scalability with exponent 0.96

Numerical chaos comb reproduces experimental results accurately

Abstract

The Multi-Armed Bandit (MAB) problem, foundational to reinforcement learning-based decision-making, addresses the challenge of maximizing rewards amidst multiple uncertain choices. While algorithmic solutions are effective, their computational efficiency diminishes with increasing problem complexity. Photonic accelerators, leveraging temporal and spatial-temporal chaos, have emerged as promising alternatives. However, despite these advancements, current approaches either compromise computation speed or amplify system complexity. In this paper, we introduce a chaotic microresonator frequency comb (chaos comb) to tackle the MAB problem, where each comb mode is assigned to a slot machine. Through a proof-of-concept experiment, we employ 44 comb modes to address an MAB with 44 slot machines, demonstrating performance competitive with both conventional software algorithms and other photonic methods. Further, the scalability of decision making is explored with up to 512 slot machines using experimentally obtained temporal chaos in different time slots. Power-law scalability is achieved with an exponent of 0.96, outperforming conventional software-based algorithms. Moreover, we find that a numerically calculated chaos comb accurately reproduces experimental results, paving the way for discussions on strategies to increase the number of slot machines.