Mitosis: Transparently Self-Replicating Page-Tables for Large-Memory Machines

TL;DR

Mitosis is a system that improves performance on large multi-socket machines by transparently replicating and migrating page-tables across sockets, reducing remote memory access latency without requiring application modifications.

Contribution

The paper introduces Mitosis, a novel approach for explicit page-table placement and migration to mitigate NUMA effects in large-memory multi-socket systems.

Findings

Up to 1.34x performance improvement with page-table replication.

Up to 3.24x performance gain during process migration across sockets.

Effective reduction of remote NUMA node accesses during page-table walks.

Abstract

Multi-socket machines with 1-100 TBs of physical memory are becoming prevalent. Applications running on multi-socket machines suffer non-uniform bandwidth and latency when accessing physical memory. Decades of research have focused on data allocation and placement policies in NUMA settings, but there have been no studies on the question of how to place page-tables amongst sockets. We make the case for explicit page-table allocation policies and show that page-table placement is becoming crucial to overall performance. We propose Mitosis to mitigate NUMA effects on page-table walks by transparently replicating and migrating page-tables across sockets without application changes. This reduces the frequency of accesses to remote NUMA nodes when performing page-table walks. Mitosis uses two components: (i) a mechanism to enable efficient page-table replication and migration; and (ii) policies for processes to efficiently manage and control page-table replication and migration. We implement Mitosis in Linux and evaluate its benefits on real hardware. Mitosis improves performance for large-scale multi-socket workloads by up to 1.34x by replicating page-tables across sockets. Moreover, it improves performance by up to 3.24x in cases when the OS migrates a process across sockets by enabling cross-socket page-table migration.