Quantum Mixed-State Self-Attention Network

TL;DR

This paper introduces QMSAN, a quantum-based self-attention network that enhances NLP tasks by leveraging quantum principles for similarity estimation and positional encoding, showing improved performance and robustness.

Contribution

The paper presents a novel quantum mixed-state self-attention mechanism and a quantum positional encoding scheme, advancing quantum NLP models with improved effectiveness and noise robustness.

Findings

QMSAN outperforms QSANN in text classification tasks.

QMSAN demonstrates robustness under various quantum noise conditions.

The model effectively captures sequence information without extra qubits.

Abstract

Attention mechanisms have revolutionized natural language processing. Combining them with quantum computing aims to further advance this technology. This paper introduces a novel Quantum Mixed-State Self-Attention Network (QMSAN) for natural language processing tasks. Our model leverages quantum computing principles to enhance the effectiveness of self-attention mechanisms. QMSAN uses a quantum attention mechanism based on mixed state, allowing for direct similarity estimation between queries and keys in the quantum domain. This approach leads to more effective attention coefficient calculations. We also propose an innovative quantum positional encoding scheme, implemented through fixed quantum gates within the circuit, improving the model's ability to capture sequence information without additional qubit resources. In numerical experiments of text classification tasks on public datasets, QMSAN outperforms Quantum Self-Attention Neural Network (QSANN). Furthermore, we demonstrate QMSAN's robustness in different quantum noise environments, highlighting its potential for near-term quantum devices.