Improving Cooperative Game Theory-based Data Valuation via Data Utility Learning

TL;DR

This paper introduces a data utility learning approach to efficiently estimate Shapley values and Least core in data valuation, overcoming computational challenges and improving accuracy in machine learning contexts.

Contribution

It proposes a novel method that learns to predict learning performance, enabling faster and more accurate data valuation via cooperative game theory.

Findings

Significant improvement in accuracy of SV and LC estimation.

Theoretical bounds relate prediction error to valuation approximation error.

Empirical results demonstrate enhanced efficiency and precision.

Abstract

The Shapley value (SV) and Least core (LC) are classic methods in cooperative game theory for cost/profit sharing problems. Both methods have recently been proposed as a principled solution for data valuation tasks, i.e., quantifying the contribution of individual datum in machine learning. However, both SV and LC suffer computational challenges due to the need for retraining models on combinatorially many data subsets. In this work, we propose to boost the efficiency in computing Shapley value or Least core by learning to estimate the performance of a learning algorithm on unseen data combinations. Theoretically, we derive bounds relating the error in the predicted learning performance to the approximation error in SV and LC. Empirically, we show that the proposed method can significantly improve the accuracy of SV and LC estimation.