Pitfalls in VM Implementation on CHERI: Lessons from Porting CRuby

TL;DR

This paper examines the unique challenges and pitfalls encountered when porting virtual machines, specifically CRuby, to the CHERI hardware architecture, highlighting issues caused by C language behaviors and proposing workarounds.

Contribution

It provides a detailed categorization of VM-specific pitfalls on CHERI, based on a case study of porting CRuby, and discusses practical workarounds and their impacts.

Findings

Most pitfalls stem from undefined behaviors in C on CHERI.

Workarounds can mitigate issues but may affect performance or safety.

Surveyed prior porting efforts to validate the proposed solutions.

Abstract

CHERI (Capability Hardware Enhanced RISC Instructions) is a novel hardware designed to address memory safety issues. By replacing traditional pointers with hardware capabilities, it enhances security in modern software systems. A Virtual Machine (VM) is one such system that can benefit from CHERI's protection, as it may contain latent memory vulnerabilities. However, developing and porting VMs to CHERI is a non-trivial task. There are many subtle pitfalls from the assumptions on the undefined behaviors of the C language made based on conventional architectures. Those assumptions conflict with CHERI's stricter memory safety model, causing unexpected failures. Although several prior works have discussed the process of porting VMs, they focus on the overall porting process instead of the pitfalls for VM implementation on CHERI. The guide for programming in CHERI exists, but it is for general programming, not addressing VM-specific issues. We have ported CRuby to CHERI as a case study and surveyed previous works on porting VMs to CHERI. We categorized and discussed the issues found based on their causes. In this paper, we illustrate the VM-specific pitfalls for each category. Most of the pitfalls arise from the undefined behaviors in the C language; in particular, implementation techniques and idioms of VMs often assume behaviors of traditional architectures that are invalid on CHERI. We also discuss workarounds for them and the impacts of those workarounds. We verified the validity of the workarounds by applying them to our CRuby port and by surveying the codebases of prior case studies. This work contributes to the body of knowledge on developing and porting VMs to CHERI and will help guide efforts toward constructing safer VMs.