Strategically Deceptive Model Deployment in Performative Prediction

TL;DR

This paper introduces Decoupled Performative Prediction (DPP), a framework modeling how institutions may strategically deceive by disclosing curated models while using different internal models, impacting user trust and system robustness.

Contribution

It formalizes the concept of strategic deception in model deployment, analyzes its optimization landscape, and proposes algorithms with convergence guarantees to exploit this deception.

Findings

DPP admits solutions with lower institutional risk than classical PP.

An algorithm with provable convergence under standard assumptions is proposed.

Self-imposed deception costs are insufficient to prevent strategic deception.

Abstract

Machine Learning systems are increasingly deployed in decision-making settings that shape user behavior and, in turn, the data on which future decisions are based. Performative Prediction (PP) formalizes this feedback loop by modeling how deployed models induce distributional shifts. It studies how to learn robust and well-performing models under such dynamics. However, existing PP frameworks typically assume that the model governing these decisions is the same model observed by users (therefore, to which they respond). In practice, deployer institutions may instead disclose curated models, while internally relying on distinct opaque models. We introduce Decoupled Performative Prediction (DPP), a framework that explicitly models mismatches between the model governing institutional decisions and the model that shapes user behavior. By analyzing the resulting optimization landscape, we show that DPP admits new different solutions that provably achieve lower risk for the institution than those under classical PP. We further propose an algorithm with provable convergence guarantees under standard assumptions, demonstrating how easy institutions can benefit from strategically deceptive deployment when they control model disclosure and users lack countervailing power. To capture the implications of such behavior, we introduce the deception cost, a quantitative measure of the degree of deception experienced by users. We study settings in which institutions incorporate this cost into the optimization process, motivated by reputational concerns or potential user abandonment, and show that such self-imposed constraints are insufficient to protect users. Overall, our results demonstrate that model disclosure is not merely an ethical consideration but a core technical design decision, underscoring the need for regulations that hold institutions accountable for deceptive deployment practices.