Protecting Confidentiality, Privacy and Integrity in Collaborative Learning

TL;DR

Citadel++ is a secure collaborative machine learning system that protects datasets, models, training code, and user privacy using TEEs and enhanced differential privacy, outperforming existing solutions significantly.

Contribution

The paper introduces Citadel++, a novel system combining TEEs and improved privacy mechanisms to safeguard assets and privacy in collaborative ML training, even against malicious code.

Findings

Outperforms state-of-the-art privacy-preserving systems by up to 543x on CPU.

Achieves high model utility while maintaining confidentiality and privacy.

Effectively protects against malicious models and training code.

Abstract

A collaboration between dataset owners and model owners is needed to facilitate effective machine learning (ML) training. During this collaboration, however, dataset owners and model owners want to protect the confidentiality of their respective assets (i.e., datasets, models and training code), with the dataset owners also caring about the privacy of individual users whose data is in their datasets. Existing solutions either provide limited confidentiality for models and training code, or suffer from privacy issues due to collusion. We present Citadel++, a collaborative ML training system designed to simultaneously protect the confidentiality of datasets, models and training code as well as the privacy of individual users. Citadel++ enhances differential privacy mechanisms to safeguard the privacy of individual user data while maintaining model utility. By employing Virtual Machine-level Trusted Execution Environments (TEEs) as well as the improved sandboxing and integrity mechanisms through OS-level techniques, Citadel++ effectively preserves the confidentiality of datasets, models and training code, and enforces our privacy mechanisms even when the models and training code have been maliciously designed. Our experiments show that Citadel++ provides model utility and performance while adhering to the confidentiality and privacy requirements of dataset owners and model owners, outperforming the state-of-the-art privacy-preserving training systems by up to 543x on CPU and 113x on GPU TEEs.