TL;DR
This paper introduces REM, a universal unlearning method for corrupted data in vision classifiers, capable of handling diverse unlearning tasks by redirecting corrupted data to dedicated neurons, outperforming prior specialized methods.
Contribution
The paper proposes a novel, general unlearning approach called REM that effectively handles various corrupted data scenarios in vision classifiers, unlike previous task-specific methods.
Findings
REM performs well across diverse unlearning tasks
Prior methods fail outside their targeted regions
REM's approach is robust to different corruption types
Abstract
Machine unlearning is studied for a multitude of tasks, but specialization of unlearning methods to particular tasks has made their systematic comparison challenging. To address this issue, we propose a conceptual space to characterize diverse corrupted data unlearning tasks in vision classifiers. This space is described by two dimensions, the discovery rate (the fraction of the corrupted data that are known at unlearning time) and the statistical regularity of the corrupted data (from random exemplars to shared concepts). Methods proposed previously have been targeted at portions of this space and-we show-fail predictably outside these regions. We propose a novel method, Redirection for Erasing Memory (REM), whose key feature is that corrupted data are redirected to dedicated neurons introduced at unlearning time and then discarded or deactivated to suppress the influence of corrupted…
Peer Reviews
Decision·ICLR 2026 Poster
(1)The authors introduce 2D taxonomy to identify statistical regularity as a missing dimension in unlearning research. (2)The authors proposed REM to redirect corrupted data into dedicated parameters and drop them, inspired by the existing method ETD. (3)REM is the only method that avoids collapse across both dimensions, compared with baseline methods.
(1)EM doubles the model size and is not viable for real-world models. From the perspective of efficiency, the core design of REM expands the model by adding a full set of new parameters. This may be feasible for small CIFAR-scale networks, but it is not realistic for large scale models. Even the VIT structure meets the OOM problem. The paper does not discuss scalability, memory cost, or efficiency, which makes REM difficult to use in real-world unlearning scenarios. (2)While the empirical resul
The introduced taxonomy, analysis different forms of unlearning tasks in the context of classification problems is a nice framing of the problem. The analysis of a set of unlearning methods, including their strengths and reasons why they fail in certain scenarios was well written and helpful to a non expert in unlearning. Authors provide sensible and well justified arguments. The proposed method heavily relies on prior work, but makes intelligent use of ideas developed in those work, which are
- One weakness refers to the inconsistent presentation quality of the paper, certain sections (such as section 4) are much better presented than others (such as section 5). The abstract and introduction in particular try to be too general, making it more difficult to understand the purpose and setting of the paper until section 2. Similarly, the paper mentioning a “universal unlearning method” yields expectations that the proposed work is not restricted to the classification problem. Certain te
This paper has clearly identified a gap in the literature of machine unlearning, where the corrupted data discovery rate and regularity can pose a challenging scenario. The proposed method shows good improvements under this corrupted data setting (Table 1).
The exact differences from ETD, and why these differences matter, is not very well explained. It seems to me that there are several differences, but I am not able to grasp well these implications, and why these differences are substantial or important. The experiment results are compared against methods that are rather outdated. SCRUB and ETD seem to be the latest works in Table 1, but they are both works from 2023. I strongly suggest that the authors compare their works with more recent public
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
