LLM-QFL: Distilling Large Language Model for Quantum Federated Learning

TL;DR

This paper introduces a novel method that combines large language models with quantum federated learning, enhancing efficiency, privacy, and scalability through model distillation and adaptive optimization techniques.

Contribution

It presents a federated fine-tuning approach that distills LLMs within QFL, enabling local adaptation, reducing communication, and improving performance with theoretical guarantees.

Findings

Significant reduction in communication rounds and faster convergence.

Enhanced scalability on resource-constrained quantum devices.

Provable guarantees for adaptive federated optimization.

Abstract

Inspired by the power of large language models (LLMs), our research adapts them to quantum federated learning (QFL) to boost efficiency and performance. We propose a federated fine-tuning method that distills an LLM within QFL, allowing each client to locally adapt the model to its own data while preserving privacy and reducing unnecessary global updates. The fine-tuned LLM also acts as a reinforcement agent, optimizing QFL by adjusting optimizer steps, cutting down communication rounds, and intelligently selecting clients. Experiments show significant efficiency gains. We pioneer a synergy between LLM and QFL, offering: i) practical efficiency: Reduced communication costs and faster convergence. ii) theoretical rigor: Provable guarantees for adaptive federated optimization. iii) scalability: PEFT methods (LoRA, QLoRA) enable deployment on resource-constrained quantum devices. Code implementation is available here 1.