Improving Large Models with Small models: Lower Costs and Better Performance

TL;DR

The paper introduces Data Shunt$^+$, a collaborative paradigm that leverages small and large models together to reduce costs and enhance performance on tasks like sentiment analysis.

Contribution

It proposes a novel collaborative framework, Data Shunt$^+$, that improves large model efficiency and effectiveness by utilizing small models for simpler subtasks.

Findings

Achieves higher accuracy with lower cost on sentiment analysis.

Reduces large model query costs to approximately 31% of original.

Better injects task-specific knowledge than fine-tuning.

Abstract

Pretrained large models (PLMs), such as ChatGPT, have demonstrated remarkable performance across diverse tasks. However, the significant computational requirements of PLMs have discouraged most product teams from running or fine-tuning them. In such cases, to harness the exceptional performance of PLMs, one must rely on expensive APIs, thereby exacerbating the economic burden. Despite the overall inferior performance of small models, in specific distributions, they can achieve comparable or even superior results. Consequently, some input can be processed exclusively by small models. On the other hand, certain tasks can be broken down into multiple subtasks, some of which can be completed without powerful capabilities. Under these circumstances, small models can handle the simple subtasks, allowing large models to focus on challenging subtasks, thus improving the performance. We propose Data Shunt$^+$ (DS$^+$), a general paradigm for collaboration of small and large models. DS$^+$ not only substantially reduces the cost associated with querying large models but also effectively improves large models' performance. For instance, ChatGPT achieves an accuracy of $94.43\%$ on Amazon Product sentiment analysis, and DS$^+$ achieves an accuracy of $95.64\%$, while the cost has been reduced to only $31.18\%$. Besides, experiments also prove that the proposed collaborative-based paradigm can better inject specific task knowledge into PLMs compared to fine-tuning.