Form and Function: Machine Unlearning as a Problem of Misaligned States

TL;DR

This paper models machine unlearning for online L-BFGS as a state alignment problem, introducing metrics and bounds to evaluate how well unlearning restores the optimizer to its counterfactual state.

Contribution

It formulates unlearning as a counterfactual state alignment task, introduces state-aware metrics, and evaluates a benchmark for deletion interventions in online L-BFGS.

Findings

Unlearning requires alignment with counterfactual optimizer states.

State-aware metrics effectively measure unlearning accuracy.

Memory-only and parameter-only corrections are evaluated against an oracle model.

Abstract

We formulate machine unlearning for online L-BFGS as a counterfactual state-alignment problem. Given an actual event stream and a deletion-edited counterfactual stream, the target of unlearning is the optimizer state that would have arisen had the deleted samples never been processed. We introduce state-aware metrics that separately measure parameter error, memory-operator error, combined state error, and update-direction error. The memory metric compares the inverse-Hessian actions induced by the o-L-BFGS memory, rather than treating curvature pairs as of finite influence. Under convexity assumptions, we derive a recursive bound on counterfactual state deviation. We then evaluate a state-aware benchmark of deletion interventions, including memory-only and parameter-only corrections, against an counterfactual oracle model. These results show that unlearning for online L-BFGS is not merely a parameter-correction problem: it requires alignment with a realizable counterfactual optimizer state.