A Framework for Evaluating Privacy-Utility Trade-off in Vertical Federated Learning

TL;DR

This paper introduces a comprehensive evaluation framework for privacy-utility trade-offs in vertical federated learning, assessing protection mechanisms against various attacks to guide practitioners in selecting suitable defenses.

Contribution

It proposes a general evaluation framework for privacy-utility trade-offs in VFL and applies it to compare protection mechanisms against multiple attacks.

Findings

Existing protections can thwart model inversion and label inference attacks.

Model completion attacks remain challenging to defend against.

The framework guides selecting effective privacy-preserving strategies.

Abstract

Federated learning (FL) has emerged as a practical solution to tackle data silo issues without compromising user privacy. One of its variants, vertical federated learning (VFL), has recently gained increasing attention as the VFL matches the enterprises' demands of leveraging more valuable features to build better machine learning models while preserving user privacy. Current works in VFL concentrate on developing a specific protection or attack mechanism for a particular VFL algorithm. In this work, we propose an evaluation framework that formulates the privacy-utility evaluation problem. We then use this framework as a guide to comprehensively evaluate a broad range of protection mechanisms against most of the state-of-the-art privacy attacks for three widely deployed VFL algorithms. These evaluations may help FL practitioners select appropriate protection mechanisms given specific requirements. Our evaluation results demonstrate that: the model inversion and most of the label inference attacks can be thwarted by existing protection mechanisms; the model completion (MC) attack is difficult to be prevented, which calls for more advanced MC-targeted protection mechanisms. Based on our evaluation results, we offer concrete advice on improving the privacy-preserving capability of VFL systems. The code is available at https://github.com/yankang18/Attack-Defense-VFL