A Probabilistic Perspective on Unlearning and Alignment for Large Language Models

TL;DR

This paper introduces a probabilistic evaluation framework for large language models, providing more accurate assessments of capabilities like unlearning and alignment compared to traditional deterministic methods.

Contribution

It presents the first formal probabilistic evaluation metrics for LLMs, improving reliability in critical tasks such as unlearning and alignment.

Findings

Probabilistic evaluations reveal limitations of deterministic assessments.

The proposed entropy-based loss improves unlearning performance.

Adaptive temperature scaling enhances probabilistic output calibration.

Abstract

Comprehensive evaluation of Large Language Models (LLMs) is an open research problem. Existing evaluations rely on deterministic point estimates generated via greedy decoding. However, we find that deterministic evaluations fail to capture the whole output distribution of a model, yielding inaccurate estimations of model capabilities. This is particularly problematic in critical contexts such as unlearning and alignment, where precise model evaluations are crucial. To remedy this, we introduce the first formal probabilistic evaluation framework for LLMs. Namely, we propose novel metrics with high probability guarantees concerning the output distribution of a model. Our metrics are application-independent and allow practitioners to make more reliable estimates about model capabilities before deployment. Our experimental analysis reveals that deterministic evaluations falsely indicate successful unlearning and alignment, whereas our probabilistic evaluations better capture model capabilities. We show how to overcome challenges associated with probabilistic outputs in a case study on unlearning by introducing (1) a novel loss based on entropy optimization, and (2) adaptive temperature scaling. We demonstrate that our approach significantly enhances unlearning in probabilistic settings on recent benchmarks. Overall, our proposed shift from point estimates to probabilistic evaluations of output distributions represents an important step toward comprehensive evaluations of LLMs. Code available at https://www.cs.cit.tum.de/daml/probabilistic-unlearning/.