Photonic reinforcement learning based on optoelectronic reservoir computing

TL;DR

This paper introduces a novel photonic reinforcement learning system using optoelectronic reservoir computing, achieving high-speed learning without traditional neural network training, demonstrated through benchmark tasks.

Contribution

It presents the first hardware implementation of photonic reservoir computing for reinforcement learning, significantly reducing learning time and enabling high-speed online learning.

Findings

Achieved reinforcement learning at several megahertz speed

Successfully performed CartPole-v0 and MountainCar-v0 tasks

First hardware demonstration of photonic reinforcement learning

Abstract

Reinforcement learning has been intensively investigated and developed in artificial intelligence in the absence of training data, such as autonomous driving vehicles, robot control, internet advertising, and elastic optical networks. However, the computational cost of reinforcement learning with deep neural networks is extremely high and reducing the learning cost is a challenging issue. We propose a photonic on-line implementation of reinforcement learning using optoelectronic delay-based reservoir computing, both experimentally and numerically. In the proposed scheme, we accelerate reinforcement learning at a rate of several megahertz because there is no required learning process for the internal connection weights in reservoir computing. We perform two benchmark tasks, CartPole-v0 and MountanCar-v0 tasks, to evaluate the proposed scheme. Our results represent the first hardware implementation of reinforcement learning based on photonic reservoir computing and pave the way for fast and efficient reinforcement learning as a novel photonic accelerator.