Small Language Models for Application Interactions: A Case Study

TL;DR

This paper demonstrates that small language models can effectively facilitate application interactions, outperforming larger models in accuracy and speed, especially when fine-tuned on limited data, with insights into system design considerations.

Contribution

It provides a case study showing small language models' effectiveness in real-world application interactions and discusses design considerations for such systems.

Findings

Small models outperform larger ones in accuracy and speed.

Fine-tuning on small datasets is effective.

Insights into system design for SLM-based applications.

Abstract

We study the efficacy of Small Language Models (SLMs) in facilitating application usage through natural language interactions. Our focus here is on a particular internal application used in Microsoft for cloud supply chain fulfilment. Our experiments show that small models can outperform much larger ones in terms of both accuracy and running time, even when fine-tuned on small datasets. Alongside these results, we also highlight SLM-based system design considerations.