Privacy-preserving design of graph neural networks with applications to vertical federated learning

TL;DR

This paper introduces VESPER, a privacy-preserving graph neural network framework for vertical federated learning, enabling secure and effective training on sensitive transaction network data.

Contribution

It proposes a novel privatization scheme called perturbed message passing (PMP) for GNNs in VFL, with solutions for both dense and sparse graphs.

Findings

VESPER achieves high-performance GNN training under privacy constraints.

The framework is effective on both public and industry datasets.

It balances privacy and utility in graph neural network training.

Abstract

The paradigm of vertical federated learning (VFL), where institutions collaboratively train machine learning models via combining each other's local feature or label information, has achieved great success in applications to financial risk management (FRM). The surging developments of graph representation learning (GRL) have opened up new opportunities for FRM applications under FL via efficiently utilizing the graph-structured data generated from underlying transaction networks. Meanwhile, transaction information is often considered highly sensitive. To prevent data leakage during training, it is critical to develop FL protocols with formal privacy guarantees. In this paper, we present an end-to-end GRL framework in the VFL setting called VESPER, which is built upon a general privatization scheme termed perturbed message passing (PMP) that allows the privatization of many popular graph neural architectures.Based on PMP, we discuss the strengths and weaknesses of specific design choices of concrete graph neural architectures and provide solutions and improvements for both dense and sparse graphs. Extensive empirical evaluations over both public datasets and an industry dataset demonstrate that VESPER is capable of training high-performance GNN models over both sparse and dense graphs under reasonable privacy budgets.