Lethean Attack: An Online Data Poisoning Technique

TL;DR

The paper introduces Lethean Attack, a new data poisoning method that causes online models to forget learned information, significantly degrading their performance during test-time training under distribution shifts.

Contribution

It presents a novel poisoning technique that induces catastrophic forgetting in online models, with theoretical backing and empirical validation against existing methods.

Findings

Lethean Attack can revert models to random guessing performance.

The attack is effective with short sample sequences.

It outperforms other forgetting-inducing sample sequences.

Abstract

Data poisoning is an adversarial scenario where an attacker feeds a specially crafted sequence of samples to an online model in order to subvert learning. We introduce Lethean Attack, a novel data poisoning technique that induces catastrophic forgetting on an online model. We apply the attack in the context of Test-Time Training, a modern online learning framework aimed for generalization under distribution shifts. We present the theoretical rationale and empirically compare it against other sample sequences that naturally induce forgetting. Our results demonstrate that using lethean attacks, an adversary could revert a test-time training model back to coin-flip accuracy performance using a short sample sequence.