ExpProof : Operationalizing Explanations for Confidential Models with ZKPs

TL;DR

This paper introduces ExpProof, a method using Zero-Knowledge Proofs to make explanations of machine learning models trustworthy and operational in adversarial settings, addressing manipulation concerns.

Contribution

It develops ZKP-compatible versions of LIME and evaluates their effectiveness on neural networks and random forests, enabling secure explanations.

Findings

ZKP-amenable LIME variants successfully operate in adversarial scenarios

The approach enhances trustworthiness of explanations in regulated environments

Code implementation is publicly available for reproducibility

Abstract

In principle, explanations are intended as a way to increase trust in machine learning models and are often obligated by regulations. However, many circumstances where these are demanded are adversarial in nature, meaning the involved parties have misaligned interests and are incentivized to manipulate explanations for their purpose. As a result, explainability methods fail to be operational in such settings despite the demand \cite{bordt2022post}. In this paper, we take a step towards operationalizing explanations in adversarial scenarios with Zero-Knowledge Proofs (ZKPs), a cryptographic primitive. Specifically we explore ZKP-amenable versions of the popular explainability algorithm LIME and evaluate their performance on Neural Networks and Random Forests. Our code is publicly available at https://github.com/emlaufer/ExpProof.