Quantum Reinforcement Learning in Non-Abelian Environments: Unveiling Novel Formulations and Quantum Advantage Exploration

TL;DR

This paper explores quantum reinforcement learning in non-commutative environments, introducing novel formulations and strategies that leverage quantum properties to potentially achieve decision-making advantages.

Contribution

It introduces new quantum RL formulations for non-Abelian environments, including a quantum Bellman equation and advantage function, expanding the theoretical framework of quantum decision processes.

Findings

Characterization of quantum agent's state space in Hilbert space

Development of a quantum Bellman equation for reward maximization

Design of a quantum advantage function exploiting quantum parallelism

Abstract

This paper delves into recent advancements in Quantum Reinforcement Learning (QRL), particularly focusing on non-commutative environments, which represent uncharted territory in this field. Our research endeavors to redefine the boundaries of decision-making by introducing formulations and strategies that harness the inherent properties of quantum systems. At the core of our investigation characterization of the agent's state space within a Hilbert space ($\mathcal{H}$). Here, quantum states emerge as complex superpositions of classical state introducing non-commutative quantum actions governed by unitary operators, necessitating a reimagining of state transitions. Complementing this framework is a refined reward function, rooted in quantum mechanics as a Hermitian operator on $\mathcal{H}$. This reward function serves as the foundation for the agent's decision-making process. By leveraging the quantum Bellman equation, we establish a methodology for maximizing expected cumulative reward over an infinite horizon, considering the entangled dynamics of quantum systems. We also connect the Quantum Bellman Equation to the Degree of Non Commutativity of the Environment, evident in Pure Algebra. We design a quantum advantage function. This ingeniously designed function exploits latent quantum parallelism inherent in the system, enhancing the agent's decision-making capabilities and paving the way for exploration of quantum advantage in uncharted territories. Furthermore, we address the significant challenge of quantum exploration directly, recognizing the limitations of traditional strategies in this complex environment.