Operationalising the Right to be Forgotten in LLMs: A Lightweight Sequential Unlearning Framework for Privacy-Aligned Deployment in Politically Sensitive Environments

TL;DR

This paper presents a lightweight sequential unlearning framework for LLMs to effectively suppress sensitive data, ensuring privacy compliance in politically sensitive deployments without compromising language capabilities.

Contribution

It introduces a novel layer-restricted negative fine-tuning method that separates retention and suppression objectives for privacy-aligned LLM deployment.

Findings

Effective suppression of sensitive data with minimal impact on accuracy and fluency

GPT-2 is more robust than DistilGPT-2 in privacy unlearning tasks

Sequential unlearning is practical for operational privacy compliance in LLMs

Abstract

Large Language Models (LLMs) are increasingly deployed in politically sensitive environments, where memorisation of personal data or confidential content raises regulatory concerns under frameworks such as the GDPR and its Right to be Forgotten. Translating such legal principles into large-scale generative systems presents significant technical challenges. We introduce a lightweight sequential unlearning framework that explicitly separates retention and suppression objectives. The method first stabilises benign capabilities through positive fine-tuning, then applies layer-restricted negative fine-tuning to suppress designated sensitive patterns while preserving general language competence. Experiments on the SemEval-2025 LLM Unlearning benchmark demonstrate effective behavioural suppression with minimal impact on factual accuracy and fluency. GPT-2 exhibits greater robustness than DistilGPT-2, highlighting the role of model capacity in privacy-aligned adaptation. We position sequential unlearning as a practical and reproducible mechanism for operationalising data erasure requirements in politically deployed LLMs.