Human-Calibrated Automated Testing and Validation of Generative Language Models

TL;DR

This paper presents HCAT, a comprehensive, human-calibrated framework for evaluating generative language models, especially RAG systems, focusing on safety, robustness, and alignment with human judgments in high-stakes domains.

Contribution

It introduces a novel multi-layered evaluation framework combining automated testing, explainable metrics, and calibration methods to improve GLM assessment reliability and transparency.

Findings

Effective calibration aligns machine and human judgments.

Robustness testing identifies vulnerabilities to adversarial inputs.

Targeted weakness analysis pinpoints specific model shortcomings.

Abstract

This paper introduces a comprehensive framework for the evaluation and validation of generative language models (GLMs), with a focus on Retrieval-Augmented Generation (RAG) systems deployed in high-stakes domains such as banking. GLM evaluation is challenging due to open-ended outputs and subjective quality assessments. Leveraging the structured nature of RAG systems, where generated responses are grounded in a predefined document collection, we propose the Human-Calibrated Automated Testing (HCAT) framework. HCAT integrates a) automated test generation using stratified sampling, b) embedding-based metrics for explainable assessment of functionality, risk and safety attributes, and c) a two-stage calibration approach that aligns machine-generated evaluations with human judgments through probability calibration and conformal prediction. In addition, the framework includes robustness testing to evaluate model performance against adversarial, out-of-distribution, and varied input conditions, as well as targeted weakness identification using marginal and bivariate analysis to pinpoint specific areas for improvement. This human-calibrated, multi-layered evaluation framework offers a scalable, transparent, and interpretable approach to GLM assessment, providing a practical and reliable solution for deploying GLMs in applications where accuracy, transparency, and regulatory compliance are paramount.