An Optical Control Environment for Benchmarking Reinforcement Learning Algorithms

TL;DR

This paper introduces an optical simulation environment for benchmarking reinforcement learning algorithms, highlighting the environment's realism and demonstrating the superior performance of off-policy RL methods over traditional control techniques.

Contribution

It presents a novel optics simulation environment for RL benchmarking and provides empirical results comparing various RL algorithms in this complex setting.

Findings

Off-policy RL algorithms outperform traditional control methods.

The environment captures nonconvexity, nonlinearity, and noise in optical systems.

Benchmark results demonstrate the effectiveness of RL in optical control.

Abstract

Deep reinforcement learning has the potential to address various scientific problems. In this paper, we implement an optics simulation environment for reinforcement learning based controllers. The environment captures the essence of nonconvexity, nonlinearity, and time-dependent noise inherent in optical systems, offering a more realistic setting. Subsequently, we provide the benchmark results of several reinforcement learning algorithms on the proposed simulation environment. The experimental findings demonstrate the superiority of off-policy reinforcement learning approaches over traditional control algorithms in navigating the intricacies of complex optical control environments. The code of the paper is available at https://github.com/Walleclipse/Reinforcement-Learning-Pulse-Stacking.