VMask: Tunable Label Privacy Protection for Vertical Federated Learning via Layer Masking

TL;DR

VMask is a novel layer masking framework for vertical federated learning that provides tunable label privacy protection against model completion attacks with minimal impact on model accuracy and significantly improved computational efficiency.

Contribution

VMask introduces a layer masking approach with a tunable privacy budget, balancing privacy and utility, and demonstrates superior privacy-utility trade-offs over existing defenses.

Findings

Effectively thwarts model completion attacks, reducing label inference accuracy to random guessing.

Maintains high model accuracy with only a 0.09% average drop in Transformer models.

Achieves up to 60,846 times faster runtime than cryptography-based methods.

Abstract

Though vertical federated learning (VFL) is generally considered to be privacy-preserving, recent studies have shown that VFL system is vulnerable to label inference attacks originating from various attack surfaces. Among these attacks, the model completion (MC) attack is currently the most powerful one. Existing defense methods against it either sacrifice model accuracy or incur impractical computational overhead. In this paper, we propose VMask, a novel label privacy protection framework designed to defend against MC attack from the perspective of layer masking. Our key insight is to disrupt the strong correlation between input data and intermediate outputs by applying the secret sharing (SS) technique to mask layer parameters in the attacker's model. We devise a strategy for selecting critical layers to mask, reducing the overhead that would arise from naively applying SS to the entire model. Moreover, VMask is the first framework to offer a tunable privacy budget to defenders, allowing for flexible control over the levels of label privacy according to actual requirements. We built a VFL system, implemented VMask on it, and extensively evaluated it using five model architectures and 13 datasets with different modalities, comparing it to 12 other defense methods. The results demonstrate that VMask achieves the best privacy-utility trade-off, successfully thwarting the MC attack (reducing the label inference accuracy to a random guessing level) while preserving model performance (e.g., in Transformer-based model, the averaged drop of VFL model accuracy is only 0.09%). VMask's runtime is up to 60,846 times faster than cryptography-based methods, and it only marginally exceeds that of standard VFL by 1.8 times in a large Transformer-based model, which is generally acceptable.