Hierarchical Dual-Strategy Unlearning for Biomedical and Healthcare Intelligence Using Imperfect and Privacy-Sensitive Medical Data

TL;DR

This paper introduces a hierarchical dual-strategy framework for selective unlearning in large language models, effectively removing sensitive medical knowledge while preserving core medical skills, with minimal parameter modifications.

Contribution

It proposes a novel combined approach using geometric-constrained gradients and concept-aware token interventions for precise knowledge unlearning in healthcare LLMs.

Findings

Achieves 82.7% forgetting rate on medical datasets.

Preserves 88.5% of original medical knowledge.

Modifies only 0.1% of model parameters.

Abstract

Large language models (LLMs) exhibit exceptional performance but pose substantial privacy risks due to training data memorization, particularly within healthcare contexts involving imperfect or privacy-sensitive patient information. We present a hierarchical dual-strategy framework for selective knowledge unlearning that precisely removes specialized knowledge while preserving fundamental medical competencies. Our approach synergistically integrates geometric-constrained gradient updates to selectively modulate target parameters with concept-aware token-level interventions that distinguish between preservation-critical and unlearning-targeted tokens via a unified four-level medical concept hierarchy. Comprehensive evaluations on the MedMCQA (surgical) and MHQA (anxiety, depression, trauma) datasets demonstrate superior performance, achieving an 82.7% forgetting rate and 88.5% knowledge preservation. Notably, our framework maintains robust privacy guarantees while requiring modification of only 0.1% of parameters, addressing critical needs for regulatory compliance, auditability, and ethical standards in clinical research.