Shapley Value on Uncertain Data

TL;DR

This paper extends the Shapley value framework to probabilistic data, providing methods to estimate the expected contribution and variance of data owners' stochastic samples, with theoretical guarantees and practical algorithms.

Contribution

It introduces a novel probabilistic Shapley value framework, deriving unbiased estimators and developing efficient Monte Carlo algorithms for data valuation under uncertainty.

Findings

Stronger accuracy-efficiency trade-offs achieved by proposed estimators.

Stratified pooled estimator significantly reduces variance.

Methods validated on synthetic and real datasets.

Abstract

The Shapley value provides a principled framework for fairly distributing rewards among participants according to their individual contributions. While prior work has applied this concept to data valuation in machine learning, existing formulations overwhelmingly assume that each participant contributes a fixed, deterministic dataset. In practice, however, data owners often provide samples drawn from underlying probabilistic distributions, introducing stochasticity into their marginal contributions and rendering the Shapley value itself a random variable. This work addresses this gap by proposing a framework for the Shapley value of probabilistic data distributions that quantifies both the expected contribution and the variance of each participant, thereby capturing uncertainty induced by random sampling. We develop theoretical and empirical methodologies for estimating these quantities: on the theoretical side, we derive unbiased estimators for the expectation and variance of the probabilistic Shapley value and analyze their statistical properties; on the empirical side, we introduce three Monte Carlo-based estimation algorithms - a baseline estimator using independent samples, a pooled estimator that improves efficiency through sample reuse, and a stratified pooled estimator that adaptively allocates sampling budget based on player-specific variability. Experiments on synthetic and real datasets demonstrate that these methods achieve strong accuracy-efficiency trade-offs, with the stratified pooled approach attaining substantial variance reduction at minimal additional cost. By extending Shapley value analysis from deterministic datasets to probabilistic data distributions, this work provides both theoretical rigor and practical tools for fair and reliable data valuation in modern stochastic data-sharing environments.