IOCA: High-Speed I/O-Aware LLC Management for Network-Centric Multi-Tenant Platform

TL;DR

This paper introduces IOCA, a novel LLC management mechanism for network-centric platforms that optimizes performance by adaptively managing LLC resources considering high-speed network I/O and DDIO effects.

Contribution

IOCA is the first LLC management approach that explicitly accounts for network I/O and DDIO, improving performance isolation and QoS in multi-tenant cloud environments.

Findings

IOCA reduces performance degradation caused by DDIO.

It adaptively allocates LLC resources based on workload demands.

Experimental results show minimal overhead and significant performance improvements.

Abstract

In modern server CPUs, last-level cache (LLC) is a critical hardware resource that exerts significant influence on the performance of the workloads, and how to manage LLC is a key to the performance isolation and QoS in the cloud with multi-tenancy. In this paper, we argue that besides CPU cores, high-speed network I/O is also important for LLC management. This is because of an Intel architectural innovation -- Data Direct I/O (DDIO) -- that directly injects the inbound I/O traffic to (part of) the LLC instead of the main memory. We summarize two problems caused by DDIO and show that (1) the default DDIO configuration may not always achieve optimal performance, (2) DDIO can decrease the performance of non-I/O workloads which share LLC with it by as high as 32%. We then present IOCA, the first LLC management mechanism for network-centric platforms that treats the I/O as the first-class citizen. IOCA monitors and analyzes the performance of the cores, LLC, and DDIO using CPU's hardware performance counters, and adaptively adjusts the number of LLC ways for DDIO or the tenants that demand more LLC capacity. In addition, IOCA dynamically chooses the tenants that share its LLC resource with DDIO, to minimize the performance interference by both the tenants and the I/O. Our experiments with multiple microbenchmarks and real-world applications in two major end-host network models demonstrate that IOCA can effectively reduce the performance degradation caused by DDIO, with minimal overhead.