Quantum Transformer: Accelerating model inference via quantum linear algebra

TL;DR

This paper explores how quantum computing can accelerate transformer-based AI models by developing quantum algorithms for key components, showing potential speedups in inference tasks with practical implications.

Contribution

It introduces quantum subroutines for transformer components and demonstrates potential quantum speedup in inference for large language models.

Findings

Quantum algorithms for self-attention and feed-forward networks

Potential quantum speedup demonstrated on open-source LLMs

Matrix norm influences quantum complexity significantly

Abstract

Powerful generative artificial intelligence from large language models (LLMs) harnesses extensive computational resources for inference. In this work, we investigate the transformer architecture, a key component of these models, under the lens of fault-tolerant quantum computing. We develop quantum subroutines to construct the building blocks in the transformer, including the self-attention, residual connection with layer normalization, and feed-forward network. As an important subroutine, we show how to efficiently implement the Hadamard product and element-wise functions of matrices on quantum computers. Our algorithm prepares an amplitude encoding of the transformer output, which can be measured for prediction or use in the next layer. We find that the matrix norm of the input sequence plays a dominant role in the quantum complexity. With numerical experiments on open-source LLMs, including for bio-informatics applications, we demonstrate the potential of a quantum speedup for transformer inference in practical regimes.