Feature Inference Attack on Shapley Values

TL;DR

This paper reveals privacy vulnerabilities in Shapley value-based interpretability methods used by major MLaaS platforms, demonstrating that private model inputs can be reconstructed through feature inference attacks, highlighting the need for privacy-preserving techniques.

Contribution

It is the first study to investigate privacy risks of Shapley values, proposing two attack models and demonstrating their effectiveness on leading cloud platforms.

Findings

Shapley value explanations can be exploited to infer private inputs.

Attack models successfully reconstruct most private features.

Leading MLaaS platforms are vulnerable to feature inference attacks.

Abstract

As a solution concept in cooperative game theory, Shapley value is highly recognized in model interpretability studies and widely adopted by the leading Machine Learning as a Service (MLaaS) providers, such as Google, Microsoft, and IBM. However, as the Shapley value-based model interpretability methods have been thoroughly studied, few researchers consider the privacy risks incurred by Shapley values, despite that interpretability and privacy are two foundations of machine learning (ML) models. In this paper, we investigate the privacy risks of Shapley value-based model interpretability methods using feature inference attacks: reconstructing the private model inputs based on their Shapley value explanations. Specifically, we present two adversaries. The first adversary can reconstruct the private inputs by training an attack model based on an auxiliary dataset and black-box access to the model interpretability services. The second adversary, even without any background knowledge, can successfully reconstruct most of the private features by exploiting the local linear correlations between the model inputs and outputs. We perform the proposed attacks on the leading MLaaS platforms, i.e., Google Cloud, Microsoft Azure, and IBM aix360. The experimental results demonstrate the vulnerability of the state-of-the-art Shapley value-based model interpretability methods used in the leading MLaaS platforms and highlight the significance and necessity of designing privacy-preserving model interpretability methods in future studies. To our best knowledge, this is also the first work that investigates the privacy risks of Shapley values.