Flexible Differentially Private Vertical Federated Learning with Adaptive Feature Embeddings

TL;DR

This paper proposes a flexible framework for vertical federated learning that balances differential privacy guarantees with task utility by decoupling privacy protection from utility optimization, using adaptive feature embeddings.

Contribution

It introduces a generic approach that applies norm clipping for privacy and adaptive embedding adjustments for utility, demonstrated through the VFL-AFE framework.

Findings

Effective privacy protection against attacks

Maintains high task utility

Applicable across various datasets and models

Abstract

The emergence of vertical federated learning (VFL) has stimulated concerns about the imperfection in privacy protection, as shared feature embeddings may reveal sensitive information under privacy attacks. This paper studies the delicate equilibrium between data privacy and task utility goals of VFL under differential privacy (DP). To address the generality issue of prior arts, this paper advocates a flexible and generic approach that decouples the two goals and addresses them successively. Specifically, we initially derive a rigorous privacy guarantee by applying norm clipping on shared feature embeddings, which is applicable across various datasets and models. Subsequently, we demonstrate that task utility can be optimized via adaptive adjustments on the scale and distribution of feature embeddings in an accuracy-appreciative way, without compromising established DP mechanisms. We concretize our observation into the proposed VFL-AFE framework, which exhibits effectiveness against privacy attacks and the capacity to retain favorable task utility, as substantiated by extensive experiments.