Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data

TL;DR

This paper introduces Compressed Vertical Federated Learning (C-VFL), a communication-efficient method for training models on vertically partitioned data, with theoretical convergence guarantees and significant communication reduction.

Contribution

It provides the first theoretical analysis of message compression effects in vertically partitioned federated learning, demonstrating convergence and practical communication savings.

Findings

Achieves over 90% reduction in communication

Proves convergence of non-convex objectives with compression

Maintains accuracy comparable to uncompressed VFL

Abstract

We propose Compressed Vertical Federated Learning (C-VFL) for communication-efficient training on vertically partitioned data. In C-VFL, a server and multiple parties collaboratively train a model on their respective features utilizing several local iterations and sharing compressed intermediate results periodically. Our work provides the first theoretical analysis of the effect message compression has on distributed training over vertically partitioned data. We prove convergence of non-convex objectives at a rate of $O(\frac{1}{\sqrt{T}})$ when the compression error is bounded over the course of training. We provide specific requirements for convergence with common compression techniques, such as quantization and top-$k$ sparsification. Finally, we experimentally show compression can reduce communication by over $90\%$ without a significant decrease in accuracy over VFL without compression.