Toleo: Scaling Freshness to Tera-scale Memory using CXL and PIM

TL;DR

Toleo introduces a scalable approach for maintaining memory freshness at tera-scale using trusted smart memory and CXL, eliminating the need for large Merkle trees and significantly reducing storage overhead.

Contribution

It proposes Toleo, a novel system that leverages trusted smart memory and CXL to efficiently ensure memory freshness at large scales without complex Merkle trees.

Findings

Enables freshness guarantees for 28 TB memory with a single 168 GB smart memory device.

Reduces version storage overhead by half through stealth versions.

Achieves 240x compression of version numbers at page granularity.

Abstract

Trusted hardware's freshness guarantee ensures that an adversary cannot replay an old value in response to a memory read request. They rely on maintaining a version number for each cache block and ensuring their integrity using a Merkle tree. However, these existing solutions protect only a small amount of main memory (few MBs), as the extraneous memory accesses to the Merkle tree increase prohibitively with the protected memory size. We present Toleo, which uses trusted smart memory connected through a secure CXL IDE network to safely store version numbers. Toleo eliminates the need for an unscalable Merkle tree to protect the integrity of version numbers by instead using smart memory as the root of trust. Additionally, Toleo ensures version confidentiality which enables stealth versions that reduce the version storage overhead in half. Furthermore, in the absence of Merkle tree imposed constraints, we effectively exploit version locality at page granularity to compress version number by a factor of 240. These space optimizations make it feasible for one 168 GB Toleo smart memory device to provide freshness to a 28 TB CXL-expanded main memory pool in a rack server for a negligible performance overhead. We analyze the benefits of Toleo using several privacy-sensitive genomics, graph, generative AI, and database workloads.