Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing

TL;DR

This paper introduces GETA, a dynamic testing method that evolves with large language models to better assess their ethical boundaries and reduce evaluation biases caused by static benchmarks.

Contribution

GETA is a novel generative evolving testing approach that dynamically generates test items tailored to LLM capabilities, addressing evaluation chronoeffect and improving assessment accuracy.

Findings

GETA can generate difficulty-tailored test items.

GETA provides more consistent evaluation results on unseen data.

GETA effectively addresses static benchmark limitations.

Abstract

Warning: Contains harmful model outputs. Despite significant advancements, the propensity of Large Language Models (LLMs) to generate harmful and unethical content poses critical challenges. Measuring value alignment of LLMs becomes crucial for their regulation and responsible deployment. Although numerous benchmarks have been constructed to assess social bias, toxicity, and ethical issues in LLMs, those static benchmarks suffer from evaluation chronoeffect, in which, as models rapidly evolve, existing benchmarks may leak into training data or become saturated, overestimating ever-developing LLMs. To tackle this problem, we propose GETA, a novel generative evolving testing approach based on adaptive testing methods in measurement theory. Unlike traditional adaptive testing methods that rely on a static test item pool, GETA probes the underlying moral boundaries of LLMs by dynamically generating test items tailored to model capability. GETA co-evolves with LLMs by learning a joint distribution of item difficulty and model value conformity, thus effectively addressing evaluation chronoeffect. We evaluated various popular LLMs with GETA and demonstrated that 1) GETA can dynamically create difficulty-tailored test items and 2) GETA's evaluation results are more consistent with models' performance on unseen OOD and i.i.d. items, laying the groundwork for future evaluation paradigms.