CooperLLM: Cloud-Edge-End Cooperative Federated Fine-tuning for LLMs via ZOO-based Gradient Correction

TL;DR

CooperLLM is a cloud-assisted federated fine-tuning framework for LLMs that combines lightweight ZOO updates on mobile devices with cloud-guided gradient correction, significantly improving efficiency and accuracy while preserving privacy.

Contribution

It introduces a novel cooperative federated fine-tuning approach that integrates ZOO on devices with cloud-based gradient rectification, addressing convergence and system bottlenecks.

Findings

Reduces on-device memory by up to 86.4%.

Accelerates convergence by 8.8 times.

Improves accuracy by up to 10 percentage points.

Abstract

Large Language Models (LLMs) perform well on many NLP tasks, but fine-tuning them on resource-constrained mobile devices is challenging due to high memory and computation costs, despite growing demands for privacy-preserving personalization. Federated Learning (FL) enables local-data training, yet existing methods either rely on memory-intensive backpropagation or use zeroth-order optimization (ZOO), which avoids backward passes but suffers from slow convergence and degraded accuracy. We propose CooperLLM, a cloud-assisted edge-end cooperative federated fine-tuning framework that combines ZOO on mobile devices with cloud-guided gradient rectification. Mobile clients perform lightweight ZOO updates on private data, while the cloud fine-tunes on auxiliary public data using backpropagation and injects guided perturbations to rectify local updates, improving convergence and accuracy without violating privacy. To address system bottlenecks, CooperLLM introduces pipeline scheduling and adaptive compression to overlap computation and communication and reduce memory usage. Experiments on multiple Transformer models and datasets show that CooperLLM reduces on-device memory by up to $86.4\%$, accelerates convergence by $8.8 \times$, and improves accuracy by up to 10 percentage points over state-of-the-art ZOO-based baselines.