Robustness and Generalization in Quantum Reinforcement Learning via Lipschitz Regularization

TL;DR

This paper introduces a Lipschitz regularization method for quantum reinforcement learning, improving policy robustness and generalization, validated through numerical experiments and curriculum learning enhancements.

Contribution

It proposes the RegQPG algorithm that incorporates Lipschitz bounds for enhanced robustness and generalization in quantum reinforcement learning.

Findings

RegQPG improves policy robustness and generalization.

Curriculum learning reduces training failures.

Numerical experiments validate practical benefits.

Abstract

Quantum machine learning leverages quantum computing to enhance accuracy and reduce model complexity compared to classical approaches, promising significant advancements in various fields. Within this domain, quantum reinforcement learning has garnered attention, often realized using variational quantum circuits to approximate the policy function. This paper addresses the robustness and generalization of quantum reinforcement learning by combining principles from quantum computing and control theory. Leveraging recent results on robust quantum machine learning, we utilize Lipschitz bounds to propose a regularized version of a quantum policy gradient approach, named the RegQPG algorithm. We show that training with RegQPG improves the robustness and generalization of the resulting policies. Furthermore, we introduce an algorithmic variant that incorporates curriculum learning, which minimizes failures during training. Our findings are validated through numerical experiments, demonstrating the practical benefits of our approach.