Quantum Circuit Structure Optimization for Quantum Reinforcement Learning

TL;DR

This paper introduces a quantum neural architecture search method to optimize the structure of parameterized quantum circuits in quantum reinforcement learning, leading to improved reward performance.

Contribution

It proposes a novel QRL-NAS algorithm that automatically searches for optimal PQC structures, enhancing the efficiency of quantum reinforcement learning models.

Findings

QRL-NAS outperforms fixed PQC structures in reward metrics.

Optimized PQC structures improve learning efficiency in QRL.

Experimental results validate the effectiveness of the proposed method.

Abstract

Reinforcement learning (RL) enables agents to learn optimal policies through environmental interaction. However, RL suffers from reduced learning efficiency due to the curse of dimensionality in high-dimensional spaces. Quantum reinforcement learning (QRL) addresses this issue by leveraging superposition and entanglement in quantum computing, allowing efficient handling of high-dimensional problems with fewer resources. QRL combines quantum neural networks (QNNs) with RL, where the parameterized quantum circuit (PQC) acts as the core computational module. The PQC performs linear and nonlinear transformations through gate operations, similar to hidden layers in classical neural networks. Previous QRL studies, however, have used fixed PQC structures based on empirical intuition without verifying their optimality. This paper proposes a QRL-NAS algorithm that integrates quantum neural architecture search (QNAS) to optimize PQC structures within QRL. Experiments demonstrate that QRL-NAS achieves higher rewards than QRL with fixed circuits, validating its effectiveness and practical utility.