Large Language Models over Networks: Collaborative Intelligence under Resource Constraints

TL;DR

This paper explores collaborative strategies for distributed large language models across devices and cloud, aiming to improve response quality amid resource constraints and network heterogeneity.

Contribution

It introduces a framework for vertical and horizontal LLM collaboration, and discusses training methods and open challenges in resource-aware distributed inference.

Findings

Proposes collaborative inference dimensions: vertical device-cloud and horizontal multi-agent.

Analyzes training approaches for routing policies and cooperation among LLMs.

Identifies key open challenges like scaling and trustworthiness in collaborative LLM systems.

Abstract

Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving. Yet cloud-based LLM services alone cannot serve a growing class of applications, including those operating under intermittent connectivity, sub-second latency budgets, data-residency constraints, or sustained high-volume inference. On-device deployment is in turn constrained by limited computation and memory. No single endpoint can deliver high-quality service across this spectrum. This article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages. Such collaboration strives for superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers. We present collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration, which can be combined into hybrid topologies in practice. We then examine learning to collaborate, addressing the training of routing policies and the development of cooperative capabilities among LLMs. Finally, we identify open research challenges including scaling under resource heterogeneity and trustworthy collaborative intelligence.