FHE-Agent: Automating CKKS Configuration for Practical Encrypted Inference via an LLM-Guided Agentic Framework

TL;DR

FHE-Agent automates the complex configuration of CKKS encryption parameters for practical encrypted inference by integrating an LLM-guided agentic framework, improving efficiency and feasibility over traditional heuristic methods.

Contribution

The paper introduces FHE-Agent, a novel framework combining LLMs and deterministic tools to automate CKKS configuration, enabling practical encrypted inference for complex neural networks.

Findings

FHE-Agent achieves better precision and lower latency than naive search methods.

It automatically finds feasible, secure configurations for complex models where previous heuristics fail.

The framework effectively decomposes parameter tuning into global and layer-wise steps.

Abstract

Fully Homomorphic Encryption (FHE), particularly the CKKS scheme, is a promising enabler for privacy-preserving MLaaS, but its practical deployment faces a prohibitive barrier: it heavily relies on domain expertise. Configuring CKKS involves a tightly coupled space of ring dimensions, modulus chains, and packing layouts. Without deep cryptographic knowledge to navigate these interactions, practitioners are restricted to compilers that rely on fixed heuristics. These "one-shot" tools often emit rigid configurations that are either severely over-provisioned in latency or fail to find a feasible solution entirely for deeper networks. We present FHE-Agent, an agentic framework that automates this expert reasoning process. By coupling a Large Language Model (LLM) controller with a deterministic tool suite, FHE-Agent decomposes the search into global parameter selection and layer-wise bottleneck repair. The agents operate within a multi-fidelity workflow, pruning invalid regimes using cheap static analysis and reserving expensive encrypted evaluations for the most promising candidates. We instantiate FHE-Agent on the Orion compiler and evaluate it on standard benchmarks (MLP, LeNet, LoLa) and deeper architectures (AlexNet). FHE-Agent consistently achieves better precision and lower latency than na\"ive search strategies. Crucially, it automatically discovers feasible, 128-bit secure configurations for complex models where baseline heuristics and one-shot prompts fail to produce a valid setup.