Prediction-sharing During Training and Inference

TL;DR

This paper analyzes different data-sharing contracts between firms engaged in competitive prediction tasks, exploring their theoretical properties and optimality within a Bayesian framework, and demonstrating their application on real loan data.

Contribution

It introduces a Bayesian framework to compare prediction-sharing contracts and identifies conditions under which each type of contract is optimal.

Findings

Prediction-sharing contracts vary in optimality depending on information and structural advantages.

Optimal contracts can be characterized as Pareto-efficient and individually rational.

Simulation on loan data illustrates the practical relevance of the theoretical results.

Abstract

Two firms are engaged in a competitive prediction task. Each firm has two sources of data -- labeled historical data and unlabeled inference-time data -- and uses the former to derive a prediction model, and the latter to make predictions on new instances. We study data-sharing contracts between the firms. The novelty of our study is to introduce and highlight the differences between contracts that share prediction models only, contracts to share inference-time predictions only, and contracts to share both. Our analysis proceeds on three levels. First, we develop a general Bayesian framework that facilitates our study. Second, we narrow our focus to two natural settings within this framework: (i) a setting in which the accuracy of each firm's prediction model is common knowledge, but the correlation between the respective models is unknown; and (ii) a setting in which two hypotheses exist regarding the optimal predictor, and one of the firms has a structural advantage in deducing it. Within these two settings we study optimal contract choice. More specifically, we find the individually rational and Pareto-optimal contracts for some notable cases, and describe specific settings where each of the different sharing contracts emerge as optimal. Finally, in the third level of our analysis we demonstrate the applicability of our concepts in a synthetic simulation using real loan data.