AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Difference

TL;DR

AdAEM is a novel, adaptive evaluation method that automatically generates diverse test questions to better distinguish LLMs' value differences, addressing limitations of static benchmarks and enabling dynamic tracking of model alignment.

Contribution

We introduce AdAEM, a self-extensible, adaptive evaluation algorithm that probes LLMs' value boundaries to generate informative, diverse test questions for better comparison.

Findings

AdAEM effectively distinguishes LLMs based on their value inclinations.

The method maximizes information gain to produce diverse, controversial topics.

AdAEM tracks value dynamics of LLMs over development stages.

Abstract

Assessing Large Language Models'(LLMs) underlying value differences enables comprehensive comparison of their misalignment, cultural adaptability, and biases. Nevertheless, current value measurement methods face the informativeness challenge: with often outdated, contaminated, or generic test questions, they can only capture the orientations on comment safety values, e.g., HHH, shared among different LLMs, leading to indistinguishable and uninformative results. To address this problem, we introduce AdAEM, a novel, self-extensible evaluation algorithm for revealing LLMs' inclinations. Distinct from static benchmarks, AdAEM automatically and adaptively generates and extends its test questions. This is achieved by probing the internal value boundaries of a diverse set of LLMs developed across cultures and time periods in an in-context optimization manner. Such a process theoretically maximizes an information-theoretic objective to extract diverse controversial topics that can provide more distinguishable and informative insights about models' value differences. In this way, AdAEM is able to co-evolve with the development of LLMs, consistently tracking their value dynamics. We use AdAEM to generate novel questions and conduct an extensive analysis, demonstrating our method's validity and effectiveness, laying the groundwork for better interdisciplinary research on LLMs' values and alignment. Codes and the generated evaluation questions are released at https://github.com/ValueCompass/AdAEM.