Quantum Decision Transformers (QDT): Synergistic Entanglement and Interference for Offline Reinforcement Learning

TL;DR

This paper introduces Quantum Decision Transformers (QDT), a quantum-inspired architecture that significantly improves offline reinforcement learning by leveraging entanglement and interference mechanisms, leading to substantial performance gains.

Contribution

The paper presents a novel quantum-inspired transformer architecture with entanglement and interference components, demonstrating their synergistic effect in enhancing offline RL performance.

Findings

Over 2,000% performance improvement over standard Decision Transformers

Quantum-inspired components exhibit strong synergy, outperforming individual parts

Identifies three computational advantages: non-local correlations, implicit ensemble, adaptive resource allocation

Abstract

Offline reinforcement learning enables policy learning from pre-collected datasets without environment interaction, but existing Decision Transformer (DT) architectures struggle with long-horizon credit assignment and complex state-action dependencies. We introduce the Quantum Decision Transformer (QDT), a novel architecture incorporating quantum-inspired computational mechanisms to address these challenges. Our approach integrates two core components: Quantum-Inspired Attention with entanglement operations that capture non-local feature correlations, and Quantum Feedforward Networks with multi-path processing and learnable interference for adaptive computation. Through comprehensive experiments on continuous control tasks, we demonstrate over 2,000\% performance improvement compared to standard DTs, with superior generalization across varying data qualities. Critically, our ablation studies reveal strong synergistic effects between quantum-inspired components: neither alone achieves competitive performance, yet their combination produces dramatic improvements far exceeding individual contributions. This synergy demonstrates that effective quantum-inspired architecture design requires holistic co-design of interdependent mechanisms rather than modular component adoption. Our analysis identifies three key computational advantages: enhanced credit assignment through non-local correlations, implicit ensemble behavior via parallel processing, and adaptive resource allocation through learnable interference. These findings establish quantum-inspired design principles as a promising direction for advancing transformer architectures in sequential decision-making, with implications extending beyond reinforcement learning to neural architecture design more broadly.