Incentivizing Truthfulness and Collaborative Fairness in Bayesian Learning

TL;DR

This paper introduces a mechanism for Bayesian collaborative learning that guarantees fairness and incentivizes truthful data sharing, addressing manipulation issues in data valuation.

Contribution

It proposes a novel mechanism combining semivalues and a validation-based data valuation function to ensure fairness and truthfulness at equilibrium.

Findings

The mechanism provably ensures collaborative fairness and truthfulness.

Validated on synthetic and real datasets with positive results.

Addresses manipulation in data sharing by incentivizing honest contributions.

Abstract

Collaborative machine learning involves training high-quality models using datasets from a number of sources. To incentivize sources to share data, existing data valuation methods fairly reward each source based on its data submitted as is. However, as these methods do not verify nor incentivize data truthfulness, the sources can manipulate their data (e.g., by submitting duplicated or noisy data) to artificially increase their valuations and rewards or prevent others from benefiting. This paper presents the first mechanism that provably ensures (F) collaborative fairness and incentivizes (T) truthfulness at equilibrium for Bayesian models. Our mechanism combines semivalues (e.g., Shapley value), which ensure fairness, and a truthful data valuation function (DVF) based on a validation set that is unknown to the sources. As semivalues are influenced by others' data, we introduce an additional condition to prove that a source can maximize its expected data values in coalitions and semivalues by submitting a dataset that captures its true knowledge. Additionally, we discuss the implications and suitable relaxations of (F) and (T) when the mediator has a limited budget for rewards or lacks a validation set. Our theoretical findings are validated on synthetic and real-world datasets.