Gold Doesn't Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information

TL;DR

This paper introduces SAL, a spectral method for removing private attribute information from neural representations, which effectively preserves main task performance and requires less guarded data, applicable to linear and nonlinear cases.

Contribution

The paper presents a novel spectral decomposition approach for attribute removal that outperforms previous methods in preserving task performance and reducing data requirements.

Findings

Better main task performance after attribute removal

Requires less guarded attribute data

Effective for both linear and nonlinear attribute removal

Abstract

We describe a simple and effective method (Spectral Attribute removaL; SAL) to remove private or guarded information from neural representations. Our method uses matrix decomposition to project the input representations into directions with reduced covariance with the guarded information rather than maximal covariance as factorization methods normally use. We begin with linear information removal and proceed to generalize our algorithm to the case of nonlinear information removal using kernels. Our experiments demonstrate that our algorithm retains better main task performance after removing the guarded information compared to previous work. In addition, our experiments demonstrate that we need a relatively small amount of guarded attribute data to remove information about these attributes, which lowers the exposure to sensitive data and is more suitable for low-resource scenarios. Code is available at https://github.com/jasonshaoshun/SAL.