Causality Analysis for Concurrent Reactive Systems (Extended Abstract)

TL;DR

This paper introduces a formal language-theoretic framework for causality analysis in concurrent reactive systems, enabling explanation of safety violations and comparison of causality notions for debugging and system design.

Contribution

It provides a unified formalization of causality notions, new definitions, and automata-based algorithms for causal analysis in concurrent reactive systems.

Findings

Unified causality framework for concurrent systems

Automata-based algorithms for causal set computation

Analysis of causality notions for debugging and liability

Abstract

We present a comprehensive language theoretic causality analysis framework for explaining safety property violations in the setting of concurrent reactive systems. Our framework allows us to uniformly express a number of causality notions studied in the areas of artificial intelligence and formal methods, as well as define new ones that are of potential interest in these areas. Furthermore, our formalization provides means for reasoning about the relationships between individual notions which have mostly been considered independently in prior work; and allows us to judge the appropriateness of the different definitions for various applications in system design. In particular, we consider causality analysis notions for debugging, error resilience, and liability resolution in concurrent reactive systems. Finally, we present automata-based algorithms for computing various causal sets based on our language-theoretic encoding, and derive the algorithmic complexities.

Full text

Causality analysis, which investigates questions of the form “Does event $e_{\_}1$ cause event $e_{\_}2$?” plays an important role in many areas of science, medicine and law. In formal methods, causality analysis has been used to determine the coverage of specifications [5] (that is, which parts of the system under scrutiny are relevant for the satisfaction of a specification), to explain counterexamples [2] (identify points in a counterexample trace that are relevant for the failure of a temporal specification), to construct fault trees [10], and to automatically refine system abstractions [4]. In artificial intelligence, causality-based explanation finding has applications in natural language processing, automated medical diagnosis, vision processing, and planning. Resolving liabilities in a legal setting often relies on establishing the causal relations between potential causes and the occurred damage [3].

Causality definitions based on counterfactuals, which are alternative scenarios where the suspected cause $e_{\_}1$ of $e_{\_}2$ did not happen, date back to [9] and have been extensively studied in philosophy [11]. In computer science, the most prominent and widely used definition of causality is that of [8], in which the authors write “… while it is hard to argue that our definition (or any other definition, for that matter) is the right definition, we show that it deals with the difficulties that have plagued other approaches in the past …”. Halpern and Pearl’s approach is based on structural equations, which describe causal dependencies between Boolean variables. We extend the Boolean study of causality to the temporal setting; specifically, we formalize notions of causality in concurrent reactive systems whose behaviors evolve over time. A concurrent reactive system is a composition of interacting components; the system behavior is determined by the repeated interaction between the components over time. Moreover, we consider the setting where component implementations are not available for analysis and the designer has only access to specifications of their expected behavior. Thus, when analyzing an error trace (an execution of the system that violates a desired system-level property), the only available information about the system consists of the components’ specifications and the observed trace.

In our framework, a concurrent reactive system $C_{\_}1\parallel C_{\_}2\parallel\ldots\parallel C_{\_}n$ is a composition of components $C_{\_}1,\ldots,C_{\_}n$. Each component $C_{\_}i$ is specified as a tuple $(X_{\_}i,\operatorname{\mathsf{inp}}(X_{\_}i),\operatorname{\mathsf{out}}(X_{\_}i),\Sigma_{\_}i,\varphi_{\_}i)$, where

• •

$X_{\_}i=\operatorname{\mathsf{inp}}(X_{\_}i)\uplus\operatorname{\mathsf{out}}(X_{\_}i)$ is the set of variables of the component, consisting of the input variables $\operatorname{\mathsf{inp}}(X_{\_}i)$ and the output variables $\operatorname{\mathsf{out}}(X_{\_}i)$ (the sets of input and output variables being disjoint);

• •

$\Sigma_{\_}i$ is the alphabet, consisting of all possible valuations of the variables $X_{\_}i$;

• •

$\varphi_{\_}i$ is a non-empty prefix-closed language over $\Sigma_{\_}i$, specifying the set of correct behaviours of $C_{\_}i$.

The composite system $C_{\_}1\parallel C_{\_}2\parallel\ldots\parallel C_{\_}n$ has an associated prefix-closed specification $\theta$ such that $\theta$ contains $\varphi_{\_}1\parallel\ldots\parallel\varphi_{\_}n$. Thus, the global requirement is more relaxed than the promised behaviors of the individual components. In other words, the system $C_{\_}1\parallel C_{\_}2\parallel\ldots\parallel C_{\_}n$ promises to implement or refine the global requirement $\theta$.

Consider a trace $\operatorname{\mathsf{tr}}$ of the system $C_{\_}1\parallel C_{\_}2\parallel\ldots\parallel C_{\_}n$ in which the system requirement $\theta$ is violated. Let $C^{\prime}_{\_}1,\ldots,C^{\prime}_{\_}k$ be the components which violate their local specifications $\varphi_{\_}1^{\prime},\ldots,\varphi_{\_}k^{\prime}$. The causality analysis problem is to determine which component set $\{D_{\_}1,\ldots,D_{\_}m\}\subseteq\{C^{\prime}_{\_}1,\ldots,C^{\prime}_{\_}k\}$ is liable for the global system requirement violation in $\operatorname{\mathsf{tr}}$. Our analysis reasons about two classes of scenarios to determine if $\mathcal{D}=\{D_{\_}1,\ldots,D_{\_}m\}$ is a cause:

• •

Fault Mitigation Capability analysis asks whether the correct behavior of the components in the set $\mathcal{D}$ is enough to mitigate the faults of all components (including those of components not in $\mathcal{D}$), by ensuring that the required system property holds.

• •

Fault Manifestation analysis asks whether the observed faulty behavior of the components in the set $\mathcal{D}$ is enough to manifest a global fault (i.e., a system behavior violating the global property), even if the components not in $\mathcal{D}$ were to behave correctly.

These two classifications parallel the classifications of [7, 6] of causes into necessary causes and sufficient causes. However, our analysis is not limited to specific definitions of counterfactual sets. In contrast, we provide a reasoning framework based on generic counterfactual sets, and introduce several natural instantiations. We demonstrate that the generality and modularity of our definition of causality allow us to seamlessly extend causality analysis to the case of heterogeneous fault models, where different components are examined under different fault scenarios. Finally, we present an automata-based method for determining various causal sets in the setting of heterogeneous component-fault models, and derive its algorithmic complexity.