Bento and the Art of Repeated Research

TL;DR

This paper evaluates Bento, a Rust-based file system framework, by attempting to reproduce its benchmarks, expanding testing with latency and throughput measurements, and analyzing bugs, highlighting Rust's safety benefits.

Contribution

The paper provides an empirical evaluation of Bento's performance and bug profile, extending previous benchmarks and analyzing Rust's impact on system safety.

Findings

Bento's performance patterns are similar but with more variability.

Journaling ext4 outperforms Bento-fs in latency.

Rust reduces memory-related bugs compared to C-based systems.

Abstract

Bento provides a new approach to developing file systems, with safety and high-velocity development in mind. This is achieved by using Rust, a modern and memory-safe systems programming language, and by providing a framework to run a single file system implementation in kernel space with the VFS or in user space with FUSE. In this paper, the benchmarking experiments from the Bento paper are repeated. We fail to exactly reproduce the results of the Bento paper, but more or less find the same patterns albeit with more outlying results. Additionally we unsuccessfully run a standardized test suite, and expand the set of experiments with latency benchmarks and throughput benchmarks using a RAM block device. The latency benchmarks show that ext4 with journaling consistently outperforms Bento-fs and the RAM throughput benchmarks show no additional consistent performance pattern. During this experimentation, a set of 12 bugs was encountered and analyzed. We find that the ratio of memory related bugs is lower than other systems programming projects that use C as opposed to Rust, thus supporting the claims of the Bento framework.