Comprehensive Kernel Safety in the Spectre Era: Mitigations and Performance Evaluation (Extended Version)

TL;DR

This paper analyzes kernel safety in the context of Spectre vulnerabilities, demonstrating the limitations of layout randomization against side-channels and speculative execution, and proposing enforcement mechanisms with performance evaluations.

Contribution

It relaxes previous assumptions on layout randomization's safety guarantees and introduces mechanisms to enforce speculative kernel safety in the Spectre era.

Findings

Layout randomization offers comparable safety with memory separation.

Side-channels and speculative execution threaten kernel safety.

Proposed enforcement mechanisms mitigate Spectre-related risks.

Abstract

The efficacy of address space layout randomization has been formally demonstrated in a shared-memory model by Abadi et al., contingent on specific assumptions about victim programs. However, modern operating systems, implementing layout randomization in the kernel, diverge from these assumptions and operate on a separate memory model with communication through system calls. In this work, we relax Abadi et al.'s language assumptions while demonstrating that layout randomization offers a comparable safety guarantee in a system with memory separation. However, in practice, speculative execution and side-channels are recognized threats to layout randomization. We show that kernel safety cannot be restored for attackers capable of using side-channels and speculative execution, and introduce enforcement mechanisms that can guarantee speculative kernel safety for safe system calls in the Spectre era. We implement three suitable mechanisms and we evaluate their performance overhead on the Linux kernel.