Pushing the Limits of In-Network Caching for Key-Value Stores

TL;DR

OrbitCache introduces a novel in-network caching architecture that leverages packet recirculation to efficiently cache variable-length key-value pairs in switch data planes, improving workload balancing and robustness.

Contribution

It proposes a new caching method that avoids hardware size limitations by continuously revisiting hot items via packet recirculation, unlike existing switch caching solutions.

Findings

Balances highly skewed workloads effectively

Robust performance under various system conditions

Successfully implemented on Intel Tofino switches

Abstract

We present OrbitCache, a new in-network caching architecture that can cache variable-length items to balance a wide range of key-value workloads. Unlike existing works, OrbitCache does not cache hot items in the switch memory. Instead, we make hot items revisit the switch data plane continuously by exploiting packet recirculation. Our approach keeps cached key-value pairs in the switch data plane while freeing them from item size limitations caused by hardware constraints. We implement an OrbitCache prototype on an Intel Tofino switch. Our experimental results show that OrbitCache can balance highly skewed workloads and is robust to various system conditions.