Confidence-Aware Sub-Structure Beam Search (CABS): Mitigating Hallucination in Structured Data Generation with Large Language Models

TL;DR

This paper introduces CABS, a confidence-aware decoding method for structured data generation with LLMs, significantly reducing hallucinations by leveraging sub-structure confidence estimates.

Contribution

It proposes the Confidence Network and CABS, a novel sub-structure beam search technique that improves the faithfulness of structured data generated by LLMs.

Findings

CABS outperforms traditional beam search by 16.7% recall at 90% precision.

Confidence Network provides targeted confidence estimates at sub-structure level.

Enhanced structured data generation reduces hallucinations and improves accuracy.

Abstract

Large Language Models (LLMs) have facilitated structured data generation, with applications in domains like tabular data, document databases, product catalogs, etc. However, concerns persist about generation veracity due to incorrect references or hallucinations, necessitating the incorporation of some form of model confidence for mitigation. Existing confidence estimation methods on LLM generations primarily focus on the confidence at the individual token level or the entire output sequence level, limiting their applicability to structured data generation, which consists of an intricate mix of both independent and correlated entries at the sub-structure level. In this paper, we first investigate confidence estimation methods for generated sub-structure-level data. We introduce the concept of Confidence Network that applies on the hidden state of the LLM transformer, as a more targeted estimate than the traditional token conditional probability. We further propose Confidence-Aware sub-structure Beam Search (CABS), a novel decoding method operating at the sub-structure level in structured data generation. CABS enhances the faithfulness of structured data generation by considering confidence scores from the Confidence Network for each sub-structure-level data and iteratively refining the prompts. Results show that CABS outperforms traditional token-level beam search for structured data generation by 16.7% Recall at 90% precision averagely on the problem of product attribute generation.