Applying sparse autoencoders to unlearn knowledge in language models

TL;DR

This paper explores the use of sparse autoencoders to selectively unlearn specific knowledge in language models, focusing on biology-related information, and compares their effectiveness to existing fine-tuning methods.

Contribution

It demonstrates that interpretable SAE features can unlearn biology knowledge with minimal side-effects, highlighting the importance of negative scaling and the limitations of feature ablation.

Findings

SAE features can unlearn biology knowledge with minimal side-effects

Negative scaling of features is necessary for effective unlearning

Current SAE techniques are less effective than fine-tuning for unlearning tasks

Abstract

We investigate whether sparse autoencoders (SAEs) can be used to remove knowledge from language models. We use the biology subset of the Weapons of Mass Destruction Proxy dataset and test on the gemma-2b-it and gemma-2-2b-it language models. We demonstrate that individual interpretable biology-related SAE features can be used to unlearn a subset of WMDP-Bio questions with minimal side-effects in domains other than biology. Our results suggest that negative scaling of feature activations is necessary and that zero ablating features is ineffective. We find that intervening using multiple SAE features simultaneously can unlearn multiple different topics, but with similar or larger unwanted side-effects than the existing Representation Misdirection for Unlearning technique. Current SAE quality or intervention techniques would need to improve to make SAE-based unlearning comparable to the existing fine-tuning based techniques.