Statistical Hypothesis Testing for Auditing Robustness in Language Models

TL;DR

This paper introduces a distribution-based hypothesis testing framework for auditing large language models, enabling robust evaluation of output changes under various perturbations with interpretable statistical measures.

Contribution

It presents a novel, model-agnostic hypothesis testing method that quantifies LLM output changes, supporting arbitrary perturbations and providing interpretable p-values and effect sizes.

Findings

Framework supports multiple perturbations with controlled error rates

Enables quantification of response changes and true/false positive rates

Demonstrates effectiveness across multiple case studies

Abstract

Consider the problem of testing whether the outputs of a large language model (LLM) system change under an arbitrary intervention, such as an input perturbation or changing the model variant. We cannot simply compare two LLM outputs since they might differ due to the stochastic nature of the system, nor can we compare the entire output distribution due to computational intractability. While existing methods for analyzing text-based outputs exist, they focus on fundamentally different problems, such as measuring bias or fairness. To this end, we introduce distribution-based perturbation analysis, a framework that reformulates LLM perturbation analysis as a frequentist hypothesis testing problem. We construct empirical null and alternative output distributions within a low-dimensional semantic similarity space via Monte Carlo sampling, enabling tractable inference without restrictive distributional assumptions. The framework is (i) model-agnostic, (ii) supports the evaluation of arbitrary input perturbations on any black-box LLM, (iii) yields interpretable p-values; (iv) supports multiple perturbations via controlled error rates; and (v) provides scalar effect sizes. We demonstrate the usefulness of the framework across multiple case studies, showing how we can quantify response changes, measure true/false positive rates, and evaluate alignment with reference models. Above all, we see this as a reliable frequentist hypothesis testing framework for LLM auditing.