Synthesis of Static Test Environments for Observing Sequence-like Behaviors in Autonomous Systems

TL;DR

This paper presents a formal method for generating static test environments for autonomous systems to verify high-level mission objectives using Kripke structures and an ILP-based algorithm.

Contribution

It introduces a novel algorithm for synthesizing minimally constrained test graphs to ensure autonomous systems meet sequence-based specifications.

Findings

Algorithm guarantees completeness and feasibility.

Synthesized test graphs are minimally constrained.

Validated on gridworld examples.

Abstract

In this paper, we investigate formal test-case generation for high-level mission objectives, specifically reachability, of autonomous systems. We use Kripke structures to represent the high-level decision-making of the agent under test and the abstraction of the test environment. First, we define the notion of a test specification, focusing on a fragment of linear temporal logic represented by sequence temporal logic formulas. Second, we formulate the problem of test graph synthesis to find a test configuration for which the agent must satisfy the test specification to satisfy its mission objectives. We an algorithm, based on network flows, for synthesizing a test graph by restricting transitions, represented by edge deletions, on the original graph induced by the Kripke structures. The algorithm synthesizes the test graph iteratively using an integer linear program. We prove completeness for our algorithm, and we show that the edge deletions in each iteration maintain feasibility of the integer linear program in the subsequent iteration. We formalize the notion of a minimally constrained test graph in terms of maximum flow, and prove the synthesized test graph to be minimally constrained. We demonstrate our algorithm on a simple graph and on gridworlds.