When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing

TL;DR

This paper introduces a hybrid classical-quantum model combining BERT with quantum temporal convolution for text classification, demonstrating improved performance and deployment feasibility on quantum and classical hardware.

Contribution

It proposes a novel quantum-enhanced BERT model with a quantum temporal convolution layer, advancing quantum language models in federated learning settings.

Findings

Achieves 1.57% and 1.52% relative improvements on intent classification datasets.

Demonstrates feasibility of deployment on quantum hardware and CPU-based systems.

Shows competitive performance in real-world spoken language datasets.

Abstract

The rapid development of quantum computing has demonstrated many unique characteristics of quantum advantages, such as richer feature representation and more secured protection on model parameters. This work proposes a vertical federated learning architecture based on variational quantum circuits to demonstrate the competitive performance of a quantum-enhanced pre-trained BERT model for text classification. In particular, our proposed hybrid classical-quantum model consists of a novel random quantum temporal convolution (QTC) learning framework replacing some layers in the BERT-based decoder. Our experiments on intent classification show that our proposed BERT-QTC model attains competitive experimental results in the Snips and ATIS spoken language datasets. Particularly, the BERT-QTC boosts the performance of the existing quantum circuit-based language model in two text classification datasets by 1.57% and 1.52% relative improvements. Furthermore, BERT-QTC can be feasibly deployed on both existing commercial-accessible quantum computation hardware and CPU-based interface for ensuring data isolation.