Differentially Private Relational Learning with Entity-level Privacy Guarantees

TL;DR

This paper develops a new differentially private learning framework tailored for relational data, addressing challenges of entity participation and sampling dependencies, with theoretical guarantees and practical experiments showing effective privacy-utility trade-offs.

Contribution

It introduces a novel sensitivity analysis, adaptive gradient clipping, and privacy amplification extension for relational learning, enabling entity-level DP guarantees in complex sampling scenarios.

Findings

Effective privacy-utility trade-offs demonstrated in experiments

New adaptive clipping scheme improves privacy guarantees

Extended privacy amplification results for coupled sampling

Abstract

Learning with relational and network-structured data is increasingly vital in sensitive domains where protecting the privacy of individual entities is paramount. Differential Privacy (DP) offers a principled approach for quantifying privacy risks, with DP-SGD emerging as a standard mechanism for private model training. However, directly applying DP-SGD to relational learning is challenging due to two key factors: (i) entities often participate in multiple relations, resulting in high and difficult-to-control sensitivity; and (ii) relational learning typically involves multi-stage, potentially coupled (interdependent) sampling procedures that make standard privacy amplification analyses inapplicable. This work presents a principled framework for relational learning with formal entity-level DP guarantees. We provide a rigorous sensitivity analysis and introduce an adaptive gradient clipping scheme that modulates clipping thresholds based on entity occurrence frequency. We also extend the privacy amplification results to a tractable subclass of coupled sampling, where the dependence arises only through sample sizes. These contributions lead to a tailored DP-SGD variant for relational data with provable privacy guarantees. Experiments on fine-tuning text encoders over text-attributed network-structured relational data demonstrate the strong utility-privacy trade-offs of our approach. Our code is available at https://github.com/Graph-COM/Node_DP.