Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMs

TL;DR

This paper investigates whether large language models can reliably evaluate the factual accuracy of generated summaries, revealing significant limitations and weak correlations with human judgments, especially for GPT-4 and PaLM-2.

Contribution

It introduces a novel approach for factuality assessment using a single LLM and benchmarks various LLMs against traditional and human evaluation methods.

Findings

GPT-3.5 shows some correlation with human judgments

GPT-4 and PaLM-2 lack significant correlation with human evaluations

Current LLMs have fundamental limitations in factuality assessment

Abstract

In recent years, Large Language Models (LLMs) have gained immense attention due to their notable emergent capabilities, surpassing those seen in earlier language models. A particularly intriguing application of LLMs is their role as evaluators for texts produced by various generative models. In this study, we delve into the potential of LLMs as reliable assessors of factual consistency in summaries generated by text-generation models. Initially, we introduce an innovative approach for factuality assessment using LLMs. This entails employing a singular LLM for the entirety of the question-answering-based factuality scoring process. Following this, we examine the efficacy of various LLMs in direct factuality scoring, benchmarking them against traditional measures and human annotations. Contrary to initial expectations, our results indicate a lack of significant correlations between factuality metrics and human evaluations, specifically for GPT-4 and PaLM-2. Notable correlations were only observed with GPT-3.5 across two factuality subcategories. These consistent findings across various factual error categories suggest a fundamental limitation in the current LLMs' capability to accurately gauge factuality. This version presents the information more concisely while maintaining the main points and findings of the original text.