Revisiting Swapping in User-space with Lightweight Threading

TL;DR

LightSwap is a user-space swapping system that significantly reduces page fault latency and boosts memcached throughput by redesigning swapping with lightweight threads and user-space I/O, avoiding kernel overhead.

Contribution

The paper introduces LightSwap, a novel user-space swapping scheme with a new page fault handling mechanism and lightweight threading, improving performance over traditional kernel-based swapping.

Findings

Reduces page fault handling latency by 3-5 times.

Increases memcached throughput by over 40%.

Supports paging with local SSDs and remote memories.

Abstract

Memory-intensive applications, such as in-memory databases, caching systems and key-value stores, are increasingly demanding larger main memory to fit their working sets. Conventional swapping can enlarge the memory capacity by paging out inactive pages to disks. However, the heavy I/O stack makes the traditional kernel-based swapping suffers from several critical performance issues. In this paper, we redesign the swapping system and propose LightSwap, an high-performance user-space swapping scheme that supports paging with both local SSDs and remote memories. First, to avoids kernel-involving, a novel page fault handling mechanism is proposed to handle page faults in user-space and further eliminates the heavy I/O stack with the help of user-space I/O drivers. Second, we co-design Lightswap with light weight thread (LWT) to improve system throughput and make it be transparent to user applications. Finally, we propose a try-catch framework in Lightswap to deal with paging errors which are exacerbated by the scaling in process technology. We implement Lightswap in our production-level system and evaluate it with YCSB workloads running on memcached. Results show that Ligthswap reduces the page faults handling latency by 3--5 times, and improves the throughput of memcached by more than 40% compared with the stat-of-art swapping systems.