Precision, Recall, and Sensitivity of Monitoring Partially Synchronous Distributed Systems

TL;DR

This paper analyzes how mismatches between monitor assumptions and actual system behavior affect the accuracy of predicate detection in partially synchronous distributed systems, highlighting a sensitive interval where errors are most likely.

Contribution

It provides analytical derivations and simulations to characterize the impact of impedance mismatch on predicate detection accuracy in distributed systems.

Findings

Small interval of hypersensitivity identified

Analytical expressions for sensitivity interval derived

Simulation results support analytical findings

Abstract

Runtime verification focuses on analyzing the execution of a given program by a monitor to determine if it is likely to violate its specifications. There is often an impedance mismatch between the assumptions/model of the monitor and that of the underlying program. This constitutes problems especially for distributed systems, where the concept of current time and state are inherently uncertain. A monitor designed with asynchronous system model assumptions may cause false-positives for a program executing in a partially synchronous system: the monitor may flag a global predicate that does not actually occur in the underlying system. A monitor designed with a partially synchronous system model assumption may cause false negatives as well as false positives for a program executing in an environment where the bounds on partial synchrony differ (albeit temporarily) from the monitor model assumptions. In this paper we analyze the effects of the impedance mismatch between the monitor and the underlying program for the detection of conjunctive predicates. We find that there is a small interval where the monitor assumptions are hypersensitive to the underlying program environment. We provide analytical derivations for this interval, and also provide simulation support for exploring the sensitivity of predicate detection to the impedance mismatch between the monitor and the program under a partially synchronous system.