UQLM: A Python Package for Uncertainty Quantification in Large Language Models

TL;DR

UQLM is a Python toolkit that applies advanced uncertainty quantification methods to detect hallucinations in large language models, improving their reliability and safety.

Contribution

The paper introduces UQLM, a comprehensive Python package that enables effective hallucination detection in LLMs using state-of-the-art uncertainty quantification techniques.

Findings

Provides response-level confidence scores for LLM outputs

Offers an easy-to-integrate solution for hallucination detection

Enhances the safety and trustworthiness of LLM applications

Abstract

Hallucinations, defined as instances where Large Language Models (LLMs) generate false or misleading content, pose a significant challenge that impacts the safety and trust of downstream applications. We introduce UQLM, a Python package for LLM hallucination detection using state-of-the-art uncertainty quantification (UQ) techniques. This toolkit offers a suite of UQ-based scorers that compute response-level confidence scores ranging from 0 to 1. This library provides an off-the-shelf solution for UQ-based hallucination detection that can be easily integrated to enhance the reliability of LLM outputs.