Rethinking How to Act: Action-Space Engineering for Reinforcement Learning-Based Circuit Routing in Distributed Quantum Systems

TL;DR

This paper introduces a reinforcement learning agent with a novel action-space formulation for optimizing quantum circuit routing across distributed quantum systems, achieving up to 35% reduction in execution time.

Contribution

It presents a new action-space formulation and masking strategies for RL agents to improve quantum circuit routing in distributed quantum computing architectures.

Findings

RL agent reduces execution time by up to 35%

New action-space formulation improves training and inference performance

Effective action-masking enhances routing decisions

Abstract

As it becomes increasingly difficult to monolithically scale a quantum processor, distributed quantum computing (DQC) offers an alternative by distributing qubits across multiple smaller interconnected quantum processor modules. In such an architecture, the challenge of quantum circuit compilation shifts from placing and routing qubits within one module to placing, routing and using the qubits efficiently across modules. In order to optimize circuit execution time, the right state-dependent networking decisions must be found, such as when and where to generate shared remote quantum states to support remote operations. Reinforcement learning (RL) provides a natural framework for this problem, generating a compilation policy that can generalize across different circuits. Building on the framework of Promponas et al. (2024), we introduce an agent that combines a novel action-space formulation with effective action-masking strategies. A comprehensive numerical comparison of the two approaches under different coupling constraints shows that our agent achieves improved training and inference performance with a relative reduction in the modeled execution time of up to 35\%.