Meeting in the Middle: A Co-Design Paradigm for FHE and AI Inference

TL;DR

This paper proposes a co-design approach combining specialized FHE schemes and optimized inference architectures to enable practical privacy-preserving AI inference in cloud environments.

Contribution

It introduces a co-design paradigm that aligns FHE scheme development with inference architecture constraints to reduce computational costs.

Findings

Identifies key optimization targets for FHE and inference architectures.

Proposes a meet-in-the-middle strategy for efficient privacy-preserving inference.

Lays out a roadmap for future joint FHE and AI inference system design.

Abstract

Modern cloud inference creates a two sided privacy problem where users reveal sensitive inputs to providers, while providers must execute proprietary model weights inside potentially leaky execution environments. Fully homomorphic encryption (FHE) offers cryptographic guarantees but remains prohibitively expensive for modern architectures. We argue that progress requires co-design where specializing FHE schemes/compilers for the static structure of inference circuits, while simultaneously constraining inference architectures to reduce dominant homomorphic cost drivers. We outline a meet in the middle agenda and concrete optimization targets on both axes.