On Evaluating the Durability of Safeguards for Open-Weight LLMs

TL;DR

This paper critically examines the effectiveness of current safeguards for open-weight LLMs, revealing evaluation challenges and emphasizing the need for more rigorous threat models to accurately assess safeguard durability.

Contribution

It highlights the difficulties in evaluating LLM safeguards and advocates for more precise, constrained threat models for better assessment accuracy.

Findings

Evaluating LLM safeguards is more challenging than it appears.

Current assessments may overestimate safeguard durability.

Rigorous, well-defined threat models are essential for meaningful evaluation.

Abstract

Stakeholders -- from model developers to policymakers -- seek to minimize the dual-use risks of large language models (LLMs). An open challenge to this goal is whether technical safeguards can impede the misuse of LLMs, even when models are customizable via fine-tuning or when model weights are fully open. In response, several recent studies have proposed methods to produce durable LLM safeguards for open-weight LLMs that can withstand adversarial modifications of the model's weights via fine-tuning. This holds the promise of raising adversaries' costs even under strong threat models where adversaries can directly fine-tune model weights. However, in this paper, we urge for more careful characterization of the limits of these approaches. Through several case studies, we demonstrate that even evaluating these defenses is exceedingly difficult and can easily mislead audiences into thinking that safeguards are more durable than they really are. We draw lessons from the evaluation pitfalls that we identify and suggest future research carefully cabin claims to more constrained, well-defined, and rigorously examined threat models, which can provide more useful and candid assessments to stakeholders.