Enabling Federated Inference via Unsupervised Consensus Embedding

TL;DR

This paper introduces CE-FI, a privacy-preserving federated inference framework that enables models to cooperate without sharing raw data or model parameters, using unsupervised consensus embedding.

Contribution

It proposes a novel consensus embedding approach allowing heterogeneous models to collaborate at inference time without sharing sensitive information.

Findings

CE-FI outperforms solo inference on CIFAR-10 and CIFAR-100.

CE-FI achieves comparable results to traditional methods with stronger sharing assumptions.

Representation alignment is identified as a key bottleneck.

Abstract

Cooperative inference across independently deployed machine learning models is increasingly desirable in distributed environments, as there is a growing need to leverage multiple models while keeping their data and model parameters private. However, existing cooperative frameworks typically rely on sharing input data, model parameters, or a common encoder, which limits their applicability in privacy-sensitive or cross-organizational settings. To address this challenge, we propose Consensus Embedding-based Federated Inference (CE-FI), a framework that enables pretrained models to cooperate at inference time without sharing model parameters or raw inputs and without assuming a common encoder. CE-FI introduces two components: a Consensus Embedding (CE) layer that maps heterogeneous intermediate representations into a common embedding space, and a Cooperative Output (CO) layer that produces predictions from these embeddings. Both layers are trained using shared unlabeled data only, so the cooperative stage does not require additional labeled data. Experiments on image classification benchmarks -- CIFAR-10 and CIFAR-100 -- under diverse non-IID conditions show that CE-FI consistently outperforms solo inference and performs comparably to conventional methods that require stronger sharing assumptions. Additional evaluations on text and time-series tasks indicate applicability beyond image classification, although performance depends on the ensemble strategy. Further analysis identifies representation alignment as the primary bottleneck.