Federated Prediction-Powered Inference from Decentralized Data

TL;DR

This paper introduces Fed-PPI, a federated learning framework that enables decentralized data to produce statistically valid inference without sharing private data, addressing data silo challenges.

Contribution

The paper proposes Fed-PPI, combining federated learning with prediction-powered inference to ensure valid statistical conclusions from decentralized, private datasets.

Findings

Fed-PPI produces valid confidence intervals in experiments.

The framework effectively handles data silos in decentralized settings.

Experimental results demonstrate the method's accuracy and reliability.

Abstract

In various domains, the increasing application of machine learning allows researchers to access inexpensive predictive data, which can be utilized as auxiliary data for statistical inference. Although such data are often unreliable compared to gold-standard datasets, Prediction-Powered Inference (PPI) has been proposed to ensure statistical validity despite the unreliability. However, the challenge of `data silos' arises when the private gold-standard datasets are non-shareable for model training, leading to less accurate predictive models and invalid inferences. In this paper, we introduces the Federated Prediction-Powered Inference (Fed-PPI) framework, which addresses this challenge by enabling decentralized experimental data to contribute to statistically valid conclusions without sharing private information. The Fed-PPI framework involves training local models on private data, aggregating them through Federated Learning (FL), and deriving confidence intervals using PPI computation. The proposed framework is evaluated through experiments, demonstrating its effectiveness in producing valid confidence intervals.