Ultrafast photonic reinforcement learning based on laser chaos

TL;DR

This paper demonstrates that laser chaos can be used for ultrafast reinforcement learning, specifically solving the multi-armed bandit problem at speeds up to 1 GHz, leveraging laser-generated randomness for decision making.

Contribution

It introduces a novel approach combining laser chaos with a simple decision algorithm to achieve ultrafast, adaptive reinforcement learning at unprecedented speeds.

Findings

Laser chaos signals sampled at 100 GSample/s enable high-speed decision making.

Optimal sampling intervals maximize decision performance.

Negative autocorrelation in laser chaos correlates with improved decision accuracy.

Abstract

Reinforcement learning involves decision making in dynamic and uncertain environments, and constitutes one important element of artificial intelligence (AI). In this paper, we experimentally demonstrate that the ultrafast chaotic oscillatory dynamics of lasers efficiently solve the multi-armed bandit problem (MAB), which requires decision making concerning a class of difficult trade-offs called the exploration-exploitation dilemma. To solve the MAB, a certain degree of randomness is required for exploration purposes. However, pseudo-random numbers generated using conventional electronic circuitry encounter severe limitations in terms of their data rate and the quality of randomness due to their algorithmic foundations. We generate laser chaos signals using a semiconductor laser sampled at a maximum rate of 100 GSample/s, and combine it with a simple decision-making principle called tug-of-war with a variable threshold, to ensure ultrafast, adaptive and accurate decision making at a maximum adaptation speed of 1 GHz. We found that decision-making performance was maximized with an optimal sampling interval, and we highlight the exact coincidence between the negative autocorrelation inherent in laser chaos and decision-making performance. This study paves the way for a new realm of ultrafast photonics in the age of AI, where the ultrahigh bandwidth of photons can provide new value.