Interpreting Negation in GPT-2: Layer- and Head-Level Causal Analysis

TL;DR

This paper investigates how GPT-2 processes negation internally by analyzing layer- and head-level causal mechanisms, revealing that negation sensitivity is concentrated in specific mid-layer attention heads and can be disrupted or restored through targeted interventions.

Contribution

It introduces a causal analysis framework for understanding negation in GPT-2, identifying key attention heads responsible for negation sensitivity and demonstrating their causal role.

Findings

Negation sensitivity is concentrated in specific mid-layer attention heads.

Ablation of these heads weakens negation sensitivity, which can be partially restored by activation rescue.

The causal patterns are consistent across different negation forms and datasets.

Abstract

Negation remains a persistent challenge for modern language models, often causing reversed meanings or factual errors. In this work, we conduct a causal analysis of how GPT-2 Small internally processes such linguistic transformations. We examine its hidden representations at both the layer and head level. Our analysis is based on a self-curated 12,000-pair dataset of matched affirmative and negated sentences, covering multiple linguistic templates and forms of negation. To quantify this behavior, we define a metric, the Negation Effect Score (NES), which measures the model's sensitivity in distinguishing between affirmative statements and their negations. We carried out two key interventions to probe causal structure. In activation patching, internal activations from affirmative sentences were inserted into their negated counterparts to see how meaning shifted. In ablation, specific attention heads were temporarily disabled to observe how logical polarity changed. Together, these steps revealed how negation signals move and evolve through GPT-2's layers. Our findings indicate that this capability is not widespread; instead, it is highly concentrated within a limited number of mid-layer attention heads, primarily within layers 4 to 6. Ablating these specific components directly disrupts the model's negation sensitivity: on our in-domain, ablation increased NES (indicating weaker negation sensitivity), and re-introducing cached affirmative activations (rescue) increased NES further, confirming that these heads carry affirmative signal rather than restoring baseline behavior. On xNot360, ablation slightly decreased NES and rescue restored performance above baseline. This pattern demonstrates that these causal patterns are consistent across various negation forms and remain detectable on the external xNot360 benchmark, though with smaller magnitude.