A Structure-Agnostic Co-Tuning Framework for LLMs and SLMs in Cloud-Edge Systems

TL;DR

This paper introduces Co-PLMs, a structure-agnostic co-tuning framework that enhances collaborative training of large and small language models across cloud and edge devices, improving performance without structural constraints.

Contribution

The paper proposes a novel structure-agnostic mutual learning framework using distilled proxy models for heterogeneous LLM and SLM collaboration.

Findings

Achieves 5.38% higher Rouge-L score

Improves 4.88% in Exact Match (EM)

Outperforms state-of-the-art methods

Abstract

The surge in intelligent applications driven by large language models (LLMs) has made it increasingly difficult for bandwidth-limited cloud servers to process extensive LLM workloads in real time without compromising user data privacy. To solve these problems, recent research has focused on constructing cloud-edge consortia that integrate server-based LLM with small language models (SLMs) on mobile edge devices. Furthermore, designing collaborative training mechanisms within such consortia to enhance inference performance has emerged as a promising research direction. However, the cross-domain deployment of SLMs, coupled with structural heterogeneity in SLMs architectures, poses significant challenges to enhancing model performance. To this end, we propose Co-PLMs, a novel co-tuning framework for collaborative training of large and small language models, which integrates the process of structure-agnostic mutual learning to realize knowledge exchange between the heterogeneous language models. This framework employs distilled proxy models (DPMs) as bridges to enable collaborative training between the heterogeneous server-based LLM and on-device SLMs, while preserving the domain-specific insights of each device. The experimental results show that Co-PLMs outperform state-of-the-art methods, achieving average increases of 5.38% in Rouge-L and 4.88% in EM.