On the interplay of Explainability, Privacy and Predictive Performance with Explanation-assisted Model Extraction

TL;DR

This paper explores how explainability, privacy, and predictive performance interact in machine learning models, especially focusing on how differential privacy can mitigate model extraction attacks facilitated by explanations.

Contribution

It investigates the trade-offs between model performance, privacy, and explainability, proposing differential privacy strategies during training and explanation generation to counteract attacks.

Findings

Differential privacy can reduce the risk of model extraction attacks.

Trade-offs exist between privacy guarantees and model accuracy.

Privacy during explanation generation impacts attack success.

Abstract

Machine Learning as a Service (MLaaS) has gained important attraction as a means for deploying powerful predictive models, offering ease of use that enables organizations to leverage advanced analytics without substantial investments in specialized infrastructure or expertise. However, MLaaS platforms must be safeguarded against security and privacy attacks, such as model extraction (MEA) attacks. The increasing integration of explainable AI (XAI) within MLaaS has introduced an additional privacy challenge, as attackers can exploit model explanations particularly counterfactual explanations (CFs) to facilitate MEA. In this paper, we investigate the trade offs among model performance, privacy, and explainability when employing Differential Privacy (DP), a promising technique for mitigating CF facilitated MEA. We evaluate two distinct DP strategies: implemented during the classification model training and at the explainer during CF generation.