Explanation Design in Strategic Learning: Sufficient Explanations that Induce Non-harmful Responses

TL;DR

This paper explores how to design explanations in algorithmic decision-making that prevent strategic agents from taking harmful actions, proposing a method that balances model accuracy with agent utility.

Contribution

It introduces a necessary condition for explanations to avoid misleading agents and demonstrates that action recommendation explanations can ensure non-harmful responses under certain assumptions.

Findings

ARexes enable safe partial model disclosure.

Proposed learning procedure jointly optimizes model and explanations.

Experiments show improved predictive performance and agent utility.

Abstract

We study explanation design in algorithmic decision making with strategic agents, individuals who may modify their inputs in response to explanations of a decision maker's (DM's) predictive model. As the demand for transparent algorithmic systems continues to grow, most prior work assumes full model disclosure as the default solution. In practice, however, DMs such as financial institutions typically disclose only partial model information via explanations. Such partial disclosure can lead agents to misinterpret the model and take actions that unknowingly harm their utility. A key open question is how DMs can communicate explanations in a way that avoids harming strategic agents, while still supporting their own decision-making goals, e.g., minimising predictive error. In this work, we analyse well-known explanation methods, and establish a necessary condition to prevent explanations from misleading agents into self-harming actions. Moreover, with a conditional homogeneity assumption, we prove that action recommendation-based explanations (ARexes) are sufficient for non-harmful responses, mirroring the revelation principle in information design. To demonstrate how ARexes can be operationalised in practice, we propose a simple learning procedure that jointly optimises the predictive model and explanation policy. Experiments on synthetic and real-world tasks show that ARexes allow the DM to optimise their model's predictive performance while preserving agents' utility, offering a more refined strategy for safe and effective partial model disclosure.