Confidence Estimation for LLM-Based Dialogue State Tracking

TL;DR

This paper explores methods to estimate and calibrate confidence scores in large language models for dialogue state tracking, aiming to improve reliability and reduce hallucinations in conversational AI.

Contribution

It provides a comprehensive evaluation of confidence estimation techniques for LLMs in dialogue systems, including novel self-probing methods for closed models and fine-tuning strategies for open models.

Findings

Fine-tuning open-weight LLMs improves confidence calibration.

Self-probing enhances confidence estimation for closed models.

Better calibration correlates with higher joint goal accuracy.

Abstract

Estimation of a model's confidence on its outputs is critical for Conversational AI systems based on large language models (LLMs), especially for reducing hallucination and preventing over-reliance. In this work, we provide an exhaustive exploration of methods, including approaches proposed for open- and closed-weight LLMs, aimed at quantifying and leveraging model uncertainty to improve the reliability of LLM-generated responses, specifically focusing on dialogue state tracking (DST) in task-oriented dialogue systems (TODS). Regardless of the model type, well-calibrated confidence scores are essential to handle uncertainties, thereby improving model performance. We evaluate four methods for estimating confidence scores based on softmax, raw token scores, verbalized confidences, and a combination of these methods, using the area under the curve (AUC) metric to assess calibration, with higher AUC indicating better calibration. We also enhance these with a self-probing mechanism, proposed for closed models. Furthermore, we assess these methods using an open-weight model fine-tuned for the task of DST, achieving superior joint goal accuracy (JGA). Our findings also suggest that fine-tuning open-weight LLMs can result in enhanced AUC performance, indicating better confidence score calibration.