Distributed Execution Indexing

TL;DR

This paper introduces distributed execution indexing for microservice applications, enabling precise fault injection at the request level by tracking inter-service RPCs across executions, addressing limitations of existing methods.

Contribution

It formally defines distributed execution indexing, proposes an instantiation that handles complex program structures, and extends a resilience testing tool for practical fault injection.

Findings

Addresses gaps in request-level fault injection techniques.

Provides a formal definition and instantiation of distributed execution indexes.

Implemented as an extension of the ilibuster{} tool for Java microservices.

Abstract

This work-in-progress report presents both the design and partial evaluation of distributed execution indexing, a technique for microservice applications that precisely identifies dynamic instances of inter-service remote procedure calls (RPCs). Such an indexing scheme is critical for request-level fault injection techniques, which aim to automatically find failure-handling bugs in microservice applications.Distributed execution indexes enable granular specification of request-level faults, while also establishing a correspondence between inter-service RPCs across multiple executions, as is required to perform a systematic search of the fault space.In this paper, we formally define the general concept of a distributed execution index, which can be parameterized on different ways of identifying an RPC in a single service. We identify an instantiation that maintains precision in the presence of a variety of program structure complexities such as loops, function indirection, and concurrency with scheduling nondeterminism. We demonstrate that this particular instantiation addresses gaps in the state-of-the-art in request-level fault injection and show that they are all special cases of distributed execution indexing. We discuss the implementation challenges and provide an implementation of distributed execution indexing as an extension of \Filibuster{}, a resilience testing tool for microservice applications for the Java programming language, which supports fault injection for gRPC and HTTP.