A Game-theoretic Understanding of Repeated Explanations in ML Models

TL;DR

This paper models the strategic interactions between ML explanation systems and end-users, including malicious actors, using game theory to understand the trade-offs and equilibrium behaviors in repeated queries.

Contribution

It introduces a formal game-theoretic framework for analyzing repeated explanations in ML, capturing strategic behaviors of both systems and users without prior assumptions.

Findings

Characterizes the equilibrium strategies in the signaling game.

Provides insights into optimal information sharing policies.

Analyzes the impact of user maliciousness on system behavior.

Abstract

This paper formally models the strategic repeated interactions between a system, comprising of a machine learning (ML) model and associated explanation method, and an end-user who is seeking a prediction/label and its explanation for a query/input, by means of game theory. In this game, a malicious end-user must strategically decide when to stop querying and attempt to compromise the system, while the system must strategically decide how much information (in the form of noisy explanations) it should share with the end-user and when to stop sharing, all without knowing the type (honest/malicious) of the end-user. This paper formally models this trade-off using a continuous-time stochastic Signaling game framework and characterizes the Markov perfect equilibrium state within such a framework.