Use Privacy in Data-Driven Systems: Theory and Experiments with Machine Learnt Programs

TL;DR

This paper introduces a formal approach to detect and repair proxy use of protected information in data-driven systems, ensuring privacy while maintaining model accuracy through analysis, detection, and transformation techniques.

Contribution

It presents a novel formal definition of proxy use, a program analysis method for detection, and a repair algorithm to eliminate inappropriate proxy use in models.

Findings

Effective detection of proxy use in social datasets

Ability to remove proxy use without significant accuracy loss

Detection surpasses existing techniques in challenging cases

Abstract

This paper presents an approach to formalizing and enforcing a class of use privacy properties in data-driven systems. In contrast to prior work, we focus on use restrictions on proxies (i.e. strong predictors) of protected information types. Our definition relates proxy use to intermediate computations that occur in a program, and identify two essential properties that characterize this behavior: 1) its result is strongly associated with the protected information type in question, and 2) it is likely to causally affect the final output of the program. For a specific instantiation of this definition, we present a program analysis technique that detects instances of proxy use in a model, and provides a witness that identifies which parts of the corresponding program exhibit the behavior. Recognizing that not all instances of proxy use of a protected information type are inappropriate, we make use of a normative judgment oracle that makes this inappropriateness determination for a given witness. Our repair algorithm uses the witness of an inappropriate proxy use to transform the model into one that provably does not exhibit proxy use, while avoiding changes that unduly affect classification accuracy. Using a corpus of social datasets, our evaluation shows that these algorithms are able to detect proxy use instances that would be difficult to find using existing techniques, and subsequently remove them while maintaining acceptable classification performance.