Fast Fourier Transform-Based Spectral and Temporal Gradient Filtering for Differential Privacy

TL;DR

This paper introduces FFTKF, a novel differentially private optimization method that uses frequency-domain filtering to improve gradient quality and accuracy in machine learning models while maintaining privacy guarantees.

Contribution

It proposes FFTKF, a frequency-domain filtering approach that enhances gradient signals under differential privacy, outperforming existing methods like DP-SGD and DiSK in accuracy.

Findings

Higher test accuracy on multiple datasets.

Reduced variance and improved privacy-utility trade-off.

Efficient $ ext{O}(d ext{log} d)$ complexity per iteration.

Abstract

Differential Privacy (DP) has emerged as a key framework for protecting sensitive data in machine learning, but standard DP-SGD often suffers from significant accuracy loss due to injected noise. To address this limitation, we introduce the FFT-Enhanced Kalman Filter (FFTKF), a differentially private optimization method that improves gradient quality while preserving $(\varepsilon, \delta)$-DP guarantees. FFTKF applies frequency-domain filtering to shift privacy noise into less informative high-frequency components, preserving the low-frequency gradient signals that carry most learning information. A scalar-gain Kalman filter with a finite-difference Hessian approximation further refines the denoised gradients. The method has per-iteration complexity $\mathcal{O}(d \log d)$ and achieves higher test accuracy than DP-SGD and DiSK on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet with CNNs, Wide ResNets, and Vision Transformers. Theoretical analysis shows that FFTKF ensures equivalent privacy while delivering a stronger privacy--utility trade-off through reduced variance and controlled bias.