Computation offloading to hardware accelerators in Intel SGX and Gramine Library OS

TL;DR

This paper extends the Gramine Library OS to support hardware accelerators within Intel SGX enclaves, enabling offloading computations to GPUs, NICs, FPGAs, and TPMs for broader application compatibility and enhanced security.

Contribution

We designed and implemented device-backed mmap and ioctl support in Gramine for SGX, allowing secure offloading to hardware accelerators and enabling application slicing between trusted and untrusted parts.

Findings

Successful implementation on Intel Media SDK workloads

Enhanced support for GPU, NIC, FPGA, and TPM offloading

Discussion of limitations and potential use cases for application slicing

Abstract

The Intel Software Guard Extensions (SGX) technology enables applications to run in an isolated SGX enclave environment, with elevated confidentiality and integrity guarantees. Gramine Library OS facilitates execution of existing unmodified applications in SGX enclaves, requiring only an accompanying manifest file that describes the application's security posture and configuration. However, Intel SGX is a CPU-only technology, thus Gramine currently supports CPU-only workloads. To enable a broader class of applications that offload computations to hardware accelerators - GPU offload, NIC offload, FPGA offload, TPM communications - Gramine must be augmented with device-backed mmap support and generic ioctl support. In this paper, we describe the design and implementation of this newly added support, the corresponding changes to the manifest-file syntax and the requisite deep copy algorithm. We evaluate our implementation on Intel Media SDK workloads and discuss the encountered caveats and limitations. Finally, we outline a use case for the presented mmap/ioctl support beyond mere device communication, namely the mechanism to slice the application into the trusted enclave part (where the core application executes) and the untrusted shared-memory part (where insecure shared libraries execute).