CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing

TL;DR

CLaRE is a lightweight, efficient technique that quantifies fact entanglement in LLMs to predict and analyze ripple effects of model edits, improving post-edit evaluation and safety.

Contribution

Introduces CLaRE, a novel, fast, and resource-efficient method to identify potential ripple effects in LLMs using representation-level entanglement analysis.

Findings

CLaRE achieves 62.2% better correlation with ripple effects than baselines.

CLaRE is 2.74 times faster and uses 2.85 times less GPU memory than previous methods.

The approach enables scalable analysis and improved preservation of factual knowledge in LLMs.

Abstract

The static knowledge representations of large language models (LLMs) inevitably become outdated or incorrect over time. While model-editing techniques offer a promising solution by modifying a model's factual associations, they often produce unpredictable ripple effects, which are unintended behavioral changes that propagate even to the hidden space. In this work, we introduce CLaRE, a lightweight representation-level technique to identify where these ripple effects may occur. Unlike prior gradient-based methods, CLaRE quantifies entanglement between facts using forward activations from a single intermediate layer, avoiding costly backward passes. To enable systematic study, we prepare and analyse a corpus of 11,427 facts drawn from three existing datasets. Using CLaRE, we compute large-scale entanglement graphs of this corpus for multiple models, capturing how local edits propagate through representational space. These graphs enable stronger preservation sets for model editing, audit trails, efficient red-teaming, and scalable post-edit evaluation. In comparison to baselines, CLaRE achieves an average of 62.2% improvement in Spearman correlation with ripple effects while being $2.74\times$ faster, and using $2.85\times$ less peak GPU memory. Besides, CLaRE requires only a fraction of the storage needed by the baselines to compute and preserve fact representations. Our entanglement graphs and corpus are available at https://github.com/manitbaser/CLaRE.