DCT-CryptoNets: Scaling Private Inference in the Frequency Domain

TL;DR

DCT-CryptoNets leverages frequency domain transforms to significantly improve the efficiency and scalability of private inference in deep neural networks using fully homomorphic encryption, enabling faster and more practical encrypted image classification.

Contribution

It introduces a novel frequency-domain approach using DCT to reduce computational costs and improve scalability of FHE-based private inference for deep learning.

Findings

Achieves up to 5.3× latency reduction compared to prior methods.

Enables inference on ImageNet within 2.5 hours, down from 12.5 hours.

Reduces error accumulation and improves scalability with image size.

Abstract

The convergence of fully homomorphic encryption (FHE) and machine learning offers unprecedented opportunities for private inference of sensitive data. FHE enables computation directly on encrypted data, safeguarding the entire machine learning pipeline, including data and model confidentiality. However, existing FHE-based implementations for deep neural networks face significant challenges in computational cost, latency, and scalability, limiting their practical deployment. This paper introduces DCT-CryptoNets, a novel approach that operates directly in the frequency-domain to reduce the burden of computationally expensive non-linear activations and homomorphic bootstrap operations during private inference. It does so by utilizing the discrete cosine transform (DCT), commonly employed in JPEG encoding, which has inherent compatibility with remote computing services where images are generally stored and transmitted in this encoded format. DCT-CryptoNets demonstrates a substantial latency reductions of up to 5.3$\times$ compared to prior work on benchmark image classification tasks. Notably, it demonstrates inference on the ImageNet dataset within 2.5 hours (down from 12.5 hours on equivalent 96-thread compute resources). Furthermore, by learning perceptually salient low-frequency information DCT-CryptoNets improves the reliability of encrypted predictions compared to RGB-based networks by reducing error accumulating homomorphic bootstrap operations. DCT-CryptoNets also demonstrates superior scalability to RGB-based networks by further reducing computational cost as image size increases. This study demonstrates a promising avenue for achieving efficient and practical private inference of deep learning models on high resolution images seen in real-world applications.