Vectorized Attention with Learnable Encoding for Quantum Transformer

TL;DR

This paper introduces the Vectorized Quantum Transformer (VQT), a novel quantum model that improves efficiency and noise resilience in quantum transformer architectures for natural language processing tasks.

Contribution

The paper proposes VQT, supporting ideal masked attention computation with quantum approximation and efficient training, reducing classical sampling overhead and enhancing practical quantum NLP applications.

Findings

Demonstrated accuracy on IBM and IonQ quantum simulators.

Achieved competitive NLP task results on high-fidelity quantum hardware.

Reduced classical sampling overhead in quantum circuit simulation.

Abstract

Vectorized quantum block encoding provides a way to embed classical data into Hilbert space, offering a pathway for quantum models, such as Quantum Transformers (QT), that replace classical self-attention with quantum circuit simulations to operate more efficiently. Current QTs rely on deep parameterized quantum circuits (PQCs), rendering them vulnerable to QPU noise, and thus hindering their practical performance. In this paper, we propose the Vectorized Quantum Transformer (VQT), a model that supports ideal masked attention matrix computation through quantum approximation simulation and efficient training via vectorized nonlinear quantum encoder, yielding shot-efficient and gradient-free quantum circuit simulation (QCS) and reduced classical sampling overhead. In addition, we demonstrate an accuracy comparison for IBM and IonQ in quantum circuit simulation and competitive results in benchmarking natural language processing tasks on IBM state-of-the-art and high-fidelity Kingston QPU. Our noise intermediate-scale quantum friendly VQT approach unlocks a novel architecture for end-to-end machine learning in quantum computing.