Ramp Up NTT in Record Time using GPU-Accelerated Algorithms and LLM-based Code Generation

TL;DR

This paper develops a GPU-optimized number theoretic transform (GNTT) for homomorphic encryption, achieving a 62x speedup, and explores large language models for automatic GPU-friendly code generation, providing insights to accelerate privacy-preserving machine learning.

Contribution

It introduces a highly optimized GPU-friendly NTT and evaluates LLMs for automatic code generation, advancing practical GPU-accelerated cryptography implementations.

Findings

GNTT achieves 62x speedup over existing methods.

DeepSeek-R1 outperforms OpenAI models in code generation.

Optimized protocols outperform LLM-generated code in efficiency.

Abstract

Homomorphic encryption (HE) is a core building block in privacy-preserving machine learning (PPML), but HE is also widely known as its efficiency bottleneck. Therefore, many GPU-accelerated cryptographic schemes have been proposed to improve the performance of HE. However, these methods often require complex modifications tailored to specific algorithms and are tightly coupled with specific GPU and operating systems. It is interesting to ask how to generally offer more practical GPU-accelerated cryptographic algorithm implementations. Given the powerful code generation capabilities of large language models (LLMs), we aim to explore their potential to automatically generate practical GPU-friendly algorithm code using CPU-friendly code. In this paper, we focus on number theoretic transform (NTT) -- the core mechanism of HE. We first develop and optimize a GPU-friendly NTT (GNTT) family that exploits PyTorch's fast matrix computation and precomputation, achieving an approximately 62x speedup -- a significant boost over existing ones. Then we explore GPU-friendly code generation using various LLMs, including DeepSeek-R1, OpenAI o1 and o3-mini. We discover many interesting findings throughout the process. For instance, somewhat surprisingly, our experiments demonstrate that DeepSeek-R1 significantly outperforms OpenAI o3-mini and o1, but still cannot beat our optimized protocol. The findings provide valuable insights for turbocharging PPML and enhancing code generation capabilities of LLMs. Codes are available at: https://github.com/LMPC-Lab/GenGPUCrypto.