Polars inside Intel SGX2 Enclaves: An Empirical Study of Confidential Analytical Query Processing

TL;DR

This study evaluates the performance of Polars DataFrame engine inside Intel SGX2 enclaves, revealing insights into overheads, query variability, and API impacts in confidential analytics workloads.

Contribution

It provides the first empirical analysis of Arrow-native DataFrame processing within SGX2 TEEs, highlighting performance behaviors and optimization considerations.

Findings

End-to-end overhead remains nearly constant at 1.49-1.56× across configurations.

Lazy execution is significantly faster than eager execution, which can fail due to memory errors.

Overheads vary across queries, with some showing pronounced run-to-run spikes.

Abstract

Trusted Execution Environments (TEEs) have renewed interest in confidential analytics, but most prior evaluations focus on SQL database engines or earlier SGX generations. This paper studies an Arrow-native DataFrame engine, Polars, running inside Intel SGX2 enclaves via Gramine on TPC-H SF30 with Azure Blob Storage. We report both the standard TPC-H power score and a query-only variant that removes table-loading time in order to separate compute overhead from data-ingestion overhead. Across four dataset-width configurations (approximately 22-73 GB), end-to-end overhead remains nearly constant at 1.49-1.56$\times$, but this composite metric obscures two distinct behaviors: query-only overhead declines from 1.51-1.52$\times$ to 1.43-1.44$\times$, whereas table-loading overhead rises from 2.27$\times$ to 4.07$\times$. We further show that overhead is not uniform across queries: for the len130 configuration, the median per-query SGX slowdown is 1.45$\times$ with a maximum of 2.57$\times$, and a small set of queries exhibits pronounced run-to-run spikes consistent with stateful EPC pressure. Finally, we compare Polars' lazy and eager APIs under the same TEE setting. Lazy execution is 2.25-2.27$\times$ faster overall, while eager execution fails with out-of-memory errors at 41 GB and above. Relative to the recent DuckDB-SGX2 study, our results suggest that SGX2 can support Arrow-native analytical processing with a similar order of security overhead, but that load-path amplification and API-level optimization are first-order determinants of end-to-end performance.