LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems

TL;DR

This paper introduces a new framework for predicting when large language models should escalate content moderation tasks to humans, improving decision accuracy and cost-efficiency in hybrid systems.

Contribution

It develops LLM Performance Predictors (LPPs) based on output uncertainty indicators, enabling effective, cost-aware escalation decisions in real-world moderation workflows.

Findings

Significant accuracy-cost trade-off improvements over existing methods

Effective across multiple state-of-the-art LLMs and tasks

Provides new insights into failure conditions and content ambiguity

Abstract

As LLMs are increasingly integrated into human-in-the-loop content moderation systems, a central challenge is deciding when their outputs can be trusted versus when escalation for human review is preferable. We propose a novel framework for supervised LLM uncertainty quantification, learning a dedicated meta-model based on LLM Performance Predictors (LPPs) derived from LLM outputs: log-probabilities, entropy, and novel uncertainty attribution indicators. We demonstrate that our method enables cost-aware selective classification in real-world human-AI workflows: escalating high-risk cases while automating the rest. Experiments across state-of-the-art LLMs, including both off-the-shelf (Gemini, GPT) and open-source (Llama, Qwen), on multimodal and multilingual moderation tasks, show significant improvements over existing uncertainty estimators in accuracy-cost trade-offs. Beyond uncertainty estimation, the LPPs enhance explainability by providing new insights into failure conditions (e.g., ambiguous content vs. under-specified policy). This work establishes a principled framework for uncertainty-aware, scalable, and responsible human-AI moderation workflows.