Mo' Memory, Mo' Problems: Stream-Native Machine Unlearning

TL;DR

This paper introduces an online machine unlearning algorithm that operates efficiently on streaming data, achieving logarithmic regret bounds and extending model lifespan without costly retraining.

Contribution

It adapts batch unlearning to an online setting with regret guarantees and constant memory, improving efficiency and model longevity in streaming environments.

Findings

Achieves logarithmic regret bounds in online unlearning.

Uses constant memory with an online L-BFGS variant.

Extends model lifespan before retraining is needed.

Abstract

Machine unlearning work assumes a static, i.i.d training environment that doesn't truly exist. Modern ML pipelines need to learn, unlearn, and predict continuously on production streams of data. We translate batch unlearning to the online setting using notions of regret, sample complexity, and deletion capacity. We tighten regret bounds to a logarithmic $\mathcal{O}(\ln{T})$, a first for a certified unlearning algorithm. When fitted with an online variant of L-BFGS optimization, the algorithm achieves state of the art regret with a constant memory footprint. Such changes extend the lifespan of an ML model before expensive retraining, making for a more efficient unlearning process.