Task-Aware Calibration: Provably Optimal Decoding in LLMs

TL;DR

This paper introduces task calibration for LLMs, aligning their output distributions in task-specific latent spaces to achieve optimal decoding and improve generation quality.

Contribution

It proposes a novel task calibration paradigm and a decision-theoretic approach for optimal decoding in LLMs, along with a new calibration metric called TCE.

Findings

MBR decoding on task-calibrated distributions is optimal.

Task calibration improves generation quality across tasks.

TCE quantifies calibration-related excess loss.

Abstract

LLM decoding often relies on the model's predictive distribution to generate an output. Consequently, misalignment with respect to the true generating distribution leads to suboptimal decisions in practice. While a natural solution is to calibrate the model's output distribution, for LLMs, this is ill-posed at the combinatorially vast level of free-form language. We address this by building on the insight that in many tasks, these free-form outputs can be interpreted in a semantically meaningful latent structure, for example, discrete class labels, integers, or sets. We introduce task calibration as a paradigm to calibrate the model's predictive distribution in the task-induced latent space. We apply a decision-theoretic result to show that Minimum Bayes Risk (MBR) decoding on the task-calibrated latent distribution is the optimal decoding strategy on latent model beliefs. Empirically, it consistently improves generation quality across different tasks and baselines. We also introduce Task Calibration Error (TCE), an application-aware calibration metric that quantifies the excess loss due to miscalibration. Our work demonstrates that task calibration enables more reliable model decisions across various tasks and applications.