Scaling Laws for Discriminative Classification in Large Language Models

TL;DR

This paper explores how scaling laws influence discriminative classification performance in large language models, demonstrating improved accuracy and efficiency for customer support tasks through model size adjustments.

Contribution

It introduces a novel approach to framing LLMs as discriminative classifiers for customer support, with empirical scaling curves and analysis of trade-offs.

Findings

Scaling curves for validation loss and top-K accuracy are established.

Offline and online experiments show significant performance improvements.

Trade-offs between model size, latency, and accuracy are discussed.

Abstract

Modern large language models (LLMs) represent a paradigm shift in what can plausibly be expected of machine learning models. The fact that LLMs can effectively generate sensible answers to a diverse range of queries suggests that they would be useful in customer support applications. While powerful, LLMs have been observed to be prone to hallucination which unfortunately makes their near term use in customer support applications challenging. To address this issue we present a system that allows us to use an LLM to augment our customer support advocates by re-framing the language modeling task as a discriminative classification task. In this framing, we seek to present the top-K best template responses for a customer support advocate to use when responding to a customer. We present the result of both offline and online experiments where we observed offline gains and statistically significant online lifts for our experimental system. Along the way, we present observed scaling curves for validation loss and top-K accuracy, resulted from model parameter ablation studies. We close by discussing the space of trade-offs with respect to model size, latency, and accuracy as well as and suggesting future applications to explore.