Model-Based Testing of Networked Applications

TL;DR

This paper introduces an automatic testing framework for networked application protocols using a domain-specific language within Coq, enabling both testing and formal verification of server behaviors, and effectively detecting protocol violations.

Contribution

It presents a novel approach combining nondeterministic server models with automated testing in Coq, bridging testing and formal verification for network protocols.

Findings

Successfully detected RFC violations in real server implementations

Automatically derived testers can handle nondeterministic server behaviors

Framework applicable to complex protocols like HTTP/1.1

Abstract

We present a principled automatic testing framework for application-layer protocols. The key innovation is a domain-specific embedded language for writing nondeterministic models of the behavior of networked servers. These models are defined within the Coq interactive theorem prover, supporting a smooth transition from testing to formal verification. Given a server model, we show how to automatically derive a tester that probes the server for unexpected behaviors. We address the uncertainties caused by both the server's internal choices and the network delaying messages nondeterministically. The derived tester accepts server implementations whose possible behaviors are a subset of those allowed by the nondeterministic model. We demonstrate the effectiveness of this framework by using it to specify and test a fragment of the HTTP/1.1 protocol, showing that the automatically derived tester can capture RFC violations in buggy server implementations, including the latest versions of Apache and Nginx.