Model-guided Fuzzing of Distributed Systems

TL;DR

This paper introduces a model-guided fuzzing approach for distributed systems that leverages abstract formal models to improve coverage and bug detection efficiency, outperforming purely random testing methods.

Contribution

The authors develop a novel coverage-guided fuzzing algorithm that uses abstract models like TLA+ to enhance testing effectiveness for distributed systems.

Findings

Higher coverage achieved compared to random exploration.

Faster detection of bugs in distributed consensus protocols.

Discovered 13 previously unknown bugs, 4 uniquely identified by model-guided fuzzing.

Abstract

We present a coverage-guided testing algorithm for distributed systems implementations. Our main innovation is the use of an abstract formal model of the system that is used to define coverage. Such abstract models are frequently developed in early phases of protocol design and verification but are infrequently used at testing time. We show that guiding random test generation using model coverage can be effective in covering interesting points in the implementation state space. We have implemented a fuzzer for distributed system implementations and abstract models written in TLA+. Our algorithm shows better coverage over purely random exploration as well as random exploration guided by different notions of scheduler coverage and mutation. In particular, we show consistently higher coverage and detect bugs faster on implementations of distributed consensus protocols such as those in Etcd-raft and RedisRaft. Moreover, we discovered 13 previously unknown bugs in their implementations, four of which could only be detected by model-guided fuzzing.