Performative Prediction with Neural Networks

TL;DR

This paper introduces a new theoretical framework for performative prediction using neural networks, relaxing previous assumptions and demonstrating stability under more natural conditions with real data experiments.

Contribution

It proposes a novel assumption on data distribution Lipschitz continuity with respect to predictions, enabling stable neural network classifiers without convexity requirements.

Findings

Neural networks can learn performatively stable classifiers under the new assumptions.

The proposed resampling procedure models realistic distribution shifts effectively.

Theoretical results are supported by experiments on real data.

Abstract

Performative prediction is a framework for learning models that influence the data they intend to predict. We focus on finding classifiers that are performatively stable, i.e. optimal for the data distribution they induce. Standard convergence results for finding a performatively stable classifier with the method of repeated risk minimization assume that the data distribution is Lipschitz continuous to the model's parameters. Under this assumption, the loss must be strongly convex and smooth in these parameters; otherwise, the method will diverge for some problems. In this work, we instead assume that the data distribution is Lipschitz continuous with respect to the model's predictions, a more natural assumption for performative systems. As a result, we are able to significantly relax the assumptions on the loss function. In particular, we do not need to assume convexity with respect to the model's parameters. As an illustration, we introduce a resampling procedure that models realistic distribution shifts and show that it satisfies our assumptions. We support our theory by showing that one can learn performatively stable classifiers with neural networks making predictions about real data that shift according to our proposed procedure.