First Experience with Real-Time Control Using Simulated VQC-Based Quantum Policies

TL;DR

This paper explores the use of variational quantum circuits for real-time control in a cart-pole system, demonstrating potential and challenges of deploying quantum policies in real-world control hardware.

Contribution

It presents the first implementation of quantum policies using VQCs in a real-time control setting, highlighting practical feasibility and current limitations.

Findings

Quantum policies successfully balanced the cart-pole system.

Local simulation met real-time control requirements.

Cloud-based quantum processing was too slow for closed-loop control.

Abstract

This paper investigates the integration of quantum computing into offline reinforcement learning and the deployment of the resulting quantum policy in a real-time control hardware realization of the cart-pole system. Variational Quantum Circuits (VQCs) are used to represent the policy. Classical model-based offline policy search was applied, in which a pure VQC with trainable input-output weights is used as a policy network instead of a classical multilayer perceptron. The goal is to evaluate the potential of deploying quantum architectures in real-world industrial control problems. The experimental results show that the investigated model-based offline policy search is able to generate quantum policies that can balance the hardware cart-pole. A latency analysis reveals that while local simulated execution meets real-time requirements, cloud-based quantum processing remains too slow for closed-loop control.