Jacobian-Guided Anisotropic Noise Reshaping for Enhancing Representation Utility under Local Differential Privacy

TL;DR

This paper introduces a Jacobian-guided anisotropic noise reshaping technique to improve data utility in local differential privacy by focusing noise reduction on task-critical subspaces identified via the Jacobian matrix.

Contribution

It proposes a novel method that selectively attenuates noise in task-relevant subspaces, enhancing utility while maintaining privacy in LDP mechanisms.

Findings

Improves data utility by approximately 20% at ε=7.5 on CIFAR-10-C.

Effectively identifies task-critical subspaces using the Jacobian matrix.

Generalizes to both linear and non-linear models.

Abstract

While Local Differential Privacy (LDP) serves as a foundational primitive for distributed data collection, its stringent noise injection requirement often leads to severe degradation in data utility. This degradation stems from the task-agnostic nature of conventional LDP mechanisms, which inject noise uniformly across all dimensions regardless of their relative importance to the downstream objective. To address this issue, we propose a novel approach that mitigates noise in task-relevant subspaces of the data representation. Our method identifies task-critical subspaces via the Jacobian matrix of the public downstream model, selectively attenuates noise along those dimensions, and reshapes the isotropic noise of standard LDP into an anisotropic distribution. This method preserves the uniform per-dimension privacy budget while heterogeneously modulating noise impact across dimensions, thereby substantially enhancing data utility. Furthermore, our approach generalizes to both linear and non-linear models and integrates seamlessly with existing mechanisms. Extensive experiments on CIFAR-10-C (Brightness corruption at the highest severity level 5) demonstrate that integrating our approach improves the utility of PrivUnit2 and PrivUnitG by approximately 20\% at $\epsilon=7.5$. The source code is available at https://github.com/ymha/jacobian-anr-ldp.