QTP-Net: A Quantum Text Pre-training Network for Natural Language Processing

TL;DR

QTP-Net introduces a quantum-enhanced pre-training approach for NLP that encodes word meanings in superposition states, significantly improving performance on sentiment analysis and word disambiguation tasks.

Contribution

It proposes a novel Quantum Text Pre-training Network combining quantum encoding with ERNIE, advancing quantum-inspired NLP models.

Findings

Improves sentiment classification accuracy by 0.024 on average

Achieves 0.784 F1 score in WSD, outperforming previous models

Outperforms classical and quantum-inspired models on benchmark datasets

Abstract

Natural Language Processing (NLP) faces challenges in the ability to quickly model polysemous words. The Grover's Algorithm (GA) is expected to solve this problem but lacks adaptability. To address the above dilemma, a Quantum Text Pre-training Network (QTP-Net) is proposed to improve the performance of NLP tasks. First, a Quantum Enhanced Pre-training Feature Embedding (QEPFE) is developed to encode multiple meanings of words into quantum superposition states and exploit adaptive GA to fast capture rich text features. Subsequently, the QEPFE is combined with the Enhanced Representation through kNowledge IntEgration (ERNIE), a pre-trained language model proposed by Baidu, to construct QTP-Net, which is evaluated on Sentiment Classification (SC) and Word Sense Disambiguation (WSD) tasks. Experiments show that in SC, the QTP-Net improves the average accuracy by 0.024 and the F1 score by 0.029 on six benchmark datasets, comprehensively outperforming both classical and quantum-inspired models. In WSD, it reaches 0.784 average F1 score, which is 0.016 higher than the sub-optimal GlossBERT, and significantly leads on SE2, SE13, and SE15. QTP-Net provides a new solution for implicit semantic modeling in NLP and lays the foundation for future research on quantum-enhanced models.