Communication-Efficient and Differentially Private Vertical Federated Learning with Zeroth-Order Optimization

TL;DR

This paper introduces DPZV, a novel zeroth-order optimization framework for vertical federated learning that enhances communication efficiency and privacy guarantees while maintaining convergence and utility.

Contribution

It proposes a new ZO-based VFL method with calibrated DP noise, reducing communication and privacy risks, and provides theoretical convergence and privacy guarantees.

Findings

Achieves better privacy-utility tradeoff than existing methods

Requires fewer communication rounds under strict privacy constraints

Provides convergence guarantees comparable to first-order methods

Abstract

Vertical Federated Learning (VFL) enables collaborative model training across feature-partitioned devices, yet its reliance on device-server information exchange introduces significant communication overhead and privacy risks. Downlink communication from the server to devices in VFL exposes gradient-related signals of the global loss that can be leveraged in inference attacks. Existing privacy-preserving VFL approaches that inject differential privacy (DP) noise on the downlink have the natural repercussion of degraded gradient quality, slowed convergence, and excessive communication rounds. In this work, we propose DPZV, a communication-efficient and differentially private ZO-VFL framework with tunable privacy guarantees. Based on zeroth-order (ZO) optimization, DPZV injects calibrated scalar-valued DP noise on the downlink, significantly reducing variance amplification while providing equivalent protection against targeted inference attacks. Through rigorous theoretical analysis, we establish convergence guarantees comparable to first-order DP-SGD, despite relying solely on ZO estimators, and prove that DPZV satisfies $(\epsilon, \delta)$-DP. Extensive experiments demonstrate that DPZV consistently achieves a superior privacy-utility tradeoff and requires fewer communication rounds than existing DP-VFL baselines under strict privacy constraints ($\epsilon \leq 10$).