Balancing Information Accuracy and Response Timeliness in Networked LLMs

TL;DR

This paper explores a networked system of specialized LLMs to improve response accuracy and timeliness, proposing an optimization framework and demonstrating that aggregation enhances accuracy, especially among similar-performing models.

Contribution

It introduces a novel networked LLM architecture with an optimization approach to balance accuracy and response time, and shows aggregation benefits in a simulated environment.

Findings

Aggregated responses outperform individual LLMs in accuracy.

Improvement is more significant among LLMs with similar standalone performance.

The proposed model effectively balances response quality and timeliness.

Abstract

Recent advancements in Large Language Models (LLMs) have transformed many fields including scientific discovery, content generation, biomedical text mining, and educational technology. However, the substantial requirements for training data, computational resources, and energy consumption pose significant challenges for their practical deployment. A promising alternative is to leverage smaller, specialized language models and aggregate their outputs to improve overall response quality. In this work, we investigate a networked LLM system composed of multiple users, a central task processor, and clusters of topic-specialized LLMs. Each user submits categorical binary (true/false) queries, which are routed by the task processor to a selected cluster of $m$ LLMs. After gathering individual responses, the processor returns a final aggregated answer to the user. We characterize both the information accuracy and response timeliness in this setting, and formulate a joint optimization problem to balance these two competing objectives. Our extensive simulations demonstrate that the aggregated responses consistently achieve higher accuracy than those of individual LLMs. Notably, this improvement is more significant when the participating LLMs exhibit similar standalone performance.