PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protection

TL;DR

This paper introduces PQLM, a portable quantum language model that enables privacy-preserving multilingual NLP by combining quantum and classical models, demonstrating comparable performance to classical models and ensuring data privacy.

Contribution

The paper presents a novel portable quantum language model framework that can be trained on private data and applied to classical downstream tasks with privacy guarantees.

Findings

PQLM achieves similar performance to classical models on key NLP metrics.

The model demonstrates effective transferability of word embeddings to classical tasks.

Ablation studies reveal factors influencing PQLM's stability and performance.

Abstract

With careful manipulation, malicious agents can reverse engineer private information encoded in pre-trained language models. Security concerns motivate the development of quantum pre-training. In this work, we propose a highly Portable Quantum Language Model (PQLM) that can easily transmit information to downstream tasks on classical machines. The framework consists of a cloud PQLM built with random Variational Quantum Classifiers (VQC) and local models for downstream applications. We demonstrate the ad hoc portability of the quantum model by extracting only the word embeddings and effectively applying them to downstream tasks on classical machines. Our PQLM exhibits comparable performance to its classical counterpart on both intrinsic evaluation (loss, perplexity) and extrinsic evaluation (multilingual sentiment analysis accuracy) metrics. We also perform ablation studies on the factors affecting PQLM performance to analyze model stability. Our work establishes a theoretical foundation for a portable quantum pre-trained language model that could be trained on private data and made available for public use with privacy protection guarantees.