Privacy-Preserving Collaborative Learning through Feature Extraction

TL;DR

This paper introduces a privacy-preserving collaborative learning framework using feature extraction methods, comparing shared and local extractors with a baseline, balancing privacy, accuracy, and computational cost.

Contribution

It proposes and evaluates two novel feature extraction-based methods for privacy-preserving collaborative learning, demonstrating their trade-offs with a baseline approach.

Findings

LTFE offers the highest privacy protection.

CTFE and LTFE achieve the best model accuracy.

SFE has the lowest computational cost.

Abstract

We propose a framework in which multiple entities collaborate to build a machine learning model while preserving privacy of their data. The approach utilizes feature embeddings from shared/per-entity feature extractors transforming data into a feature space for cooperation between entities. We propose two specific methods and compare them with a baseline method. In Shared Feature Extractor (SFE) Learning, the entities use a shared feature extractor to compute feature embeddings of samples. In Locally Trained Feature Extractor (LTFE) Learning, each entity uses a separate feature extractor and models are trained using concatenated features from all entities. As a baseline, in Cooperatively Trained Feature Extractor (CTFE) Learning, the entities train models by sharing raw data. Secure multi-party algorithms are utilized to train models without revealing data or features in plain text. We investigate the trade-offs among SFE, LTFE, and CTFE in regard to performance, privacy leakage (using an off-the-shelf membership inference attack), and computational cost. LTFE provides the most privacy, followed by SFE, and then CTFE. Computational cost is lowest for SFE and the relative speed of CTFE and LTFE depends on network architecture. CTFE and LTFE provide the best accuracy. We use MNIST, a synthetic dataset, and a credit card fraud detection dataset for evaluations.