Responsibility Analysis by Abstract Interpretation
Chaoqiang Deng, Patrick Cousot
TL;DR
This paper introduces a new responsibility analysis method based on event trace semantics, offering more precise identification of responsible entities in program behaviors, with applications in security and scientific fields.
Contribution
It proposes a novel responsibility definition using event trace semantics that improves precision and considers observer cognizance, advancing static program analysis techniques.
Findings
More precise responsibility identification than classical methods
Incorporates observer cognizance into responsibility analysis
Applicable to program security and scientific analysis
Abstract
Given a behavior of interest in the program, statically determining the corresponding responsible entity is a task of critical importance, especially in program security. Classical static analysis techniques (e.g. dependency analysis, taint analysis, slicing, etc.) assist programmers in narrowing down the scope of responsibility, but none of them can explicitly identify the responsible entity. Meanwhile, the causality analysis is generally not pertinent for analyzing programs, and the structural equations model (SEM) of actual causality misses some information inherent in programs, making its analysis on programs imprecise. In this paper, a novel definition of responsibility based on the abstraction of event trace semantics is proposed, which can be applied in program security and other scientific fields. Briefly speaking, an entity ER is responsible for behavior B, if and only if ER is…
Click any figure to enlarge with its caption.
Figure 1
Figure 2Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Testing and Debugging Techniques · Security and Verification in Computing · Advanced Malware Detection Techniques
11institutetext: Computer Science Department, New York University, USA
11email: {deng,pcousot}@cs.nyu.edu
Responsibility Analysis
by Abstract Interpretation
Chaoqiang Deng
Patrick Cousot
Abstract
Given a behavior of interest in the program, statically determining the corresponding responsible entity is a task of critical importance, especially in program security. Classical static analysis techniques (e.g. dependency analysis, taint analysis, slicing, etc.) assist programmers in narrowing down the scope of responsibility, but none of them can explicitly identify the responsible entity. Meanwhile, the causality analysis is generally not pertinent for analyzing programs, and the structural equations model (SEM) of actual causality misses some information inherent in programs, making its analysis on programs imprecise. In this paper, a novel definition of responsibility based on the abstraction of event trace semantics is proposed, which can be applied in program security and other scientific fields. Briefly speaking, an entity is responsible for behavior , if and only if is free to choose its input value, and such a choice is the first one that ensures the occurrence of in the forthcoming execution. Compared to current analysis methods, the responsibility analysis is more precise. In addition, our definition of responsibility takes into account the cognizance of the observer, which, to the best of our knowledge, is a new innovative idea in program analysis.
Keywords:
Responsibility Abstract Interpretation Static Analysis Dependency Causality Program Security
1 Introduction
For any behavior of interest, especially potentially insecure behaviors in the program, it is essential to determine the corresponding responsible entity, or say, the root cause. Contrary to accountability mechanisms [46, 26, 16] that track down perpetrators after the fact, the goal of this paper is to detect the responsible entity and configure its permission before deploying the program, which is important for safety and security critical systems. Due to the massive scale of modern software, it is virtually impossible to identify the responsible entity manually. The only solution is to design a static analysis of responsibility, which can examine all possible executions of a program without executing them.
The cornerstone of designing such an analysis is to define responsibility in programming languages. It is surprising to notice that, although the concepts of causality and responsibility have been long studied in various contexts (law sciences [40], artificial intelligence [33], statistical and quantum mechanics, biology, social sciences, etc. [5]), none of these definitions is fully pertinent for programming languages. Take the actual cause [21, 22] as an example, its structural equations model (SEM) [11] is not suitable for representing programs: the value of each endogenous variable in the model is fixed once it is set by the equations or some external action, while the value of program variables can be assigned for unbounded number of times during the execution. In addition, the SEM cannot make use of the temporal information or whether an entity is free to make choices, which plays an indispensable role in determining responsibility.
There do exist techniques analyzing the influence relationships in programs, such as dependency analysis [1, 8, 42], taint analysis [34] and program slicing [45], which help in narrowing down the scope of possible locations of responsible entity. However, no matter whether adopting semantic or syntactic methods, these techniques are not precise enough to explicitly identify responsibility.
To solve the above problems, we propose a novel definition of responsibility based on the event trace semantics, which is expressive and generic to handle computer programs and other scientific fields. Roughly speaking, an entity is responsible for a given behavior in a certain trace, if and only if can choose various values at its discretion (e.g. inputs from external subjects), and such a choice is the first one that guarantees the occurrence of in that trace. Such a definition of responsibility is an abstract interpretation [12, 13] of event trace semantics, taking into account both the temporal ordering of events and the information regarding whether an entity is free to choose its value. Moreover, an innovative idea of cognizance is adopted in this definition, which allows analyzing responsibility from the perspective of various observers. Compared to current techniques, our definition of responsibility is more generic and precise.
The applications of responsibility analysis are pervasive. Although an implementation of responsibility analyzer is not provided here, we have demonstrated its effectiveness by examples including access control, “negative balance” and information leakage. In addition, due to the page limit, a sound framework of abstract responsibility analysis is sketched in the appendix 0.B, which is the basis of implementing a responsibility analyzer. It is guaranteed that the entities that are found definitely responsible in the abstract analysis are definitely responsible in the concrete, while those not found potentially responsible in the abstract analysis are definitely not responsible in the concrete.
To summarize, the main contributions of this work are: (1) a completely new definition of responsibility, which is based on the abstract interpretation of event trace semantics, (2) the adoption of observers’ cognizance in program analysis for the first time, (3) various examples of responsibility analysis, and (4) a sound framework for the abstract static analysis of responsibility.
In the following, section 2 discusses the distinctions between responsibility and current techniques via an example, and sketches the framework of responsibility analysis. Section 3 formally defines responsibility as an abstraction of event trace semantics. Section 4 exemplifies the applications of responsibility analysis. Section 5 summarizes the related work.
2 A Glance at Responsibility
Given a behavior of interest (e.g. security policy violation), the objective of responsibility analysis is to automatically determine which entity in the system has the primary control over that behavior. Then security admins could decide either to keep or to deny the responsible entity’s permission to perform the behavior of interest. Take the information leakage in a social network as an example: if the information’s owner is responsible for the leakage (e.g. a user shares his picture with friends), then it is safe to keep its permission to perform such a behavior; otherwise, if anyone else is responsible for the leakage, it could be a malicious attacker and its permission to do so shall be removed. Such human decisions can only be done manually and are beyond the scope of this paper. In addition, it is worthwhile to note that responsibility analysis is not the same as program debugging, since the analyzed program code is presumed to be unmodifiable and the only possible change is on the permissions granted to entities in the system.
In order to give an informal introduction to responsibility, as well as its main distinctions with dependency, causality and other techniques in detecting causes, this section starts with a simple example, which is used throughout the paper.
2.1 Discussion of an Access Control Program Example
Example 1 (Access Control)
Consider the program in Fig. 1, which essentially can be interpreted as an access control program for an object (e.g. a secret file), such that can be read if and only if both two admins approve the access and the permission type of from system settings is greater than or equal to “read”: the first two inputs correspond to the decisions of two independent admins, where 1 represents approving the access to , and 0 represents rejecting the access; the third input stored in typ represents the permission type of specified in the system settings, where 1 represents “read”, 2 represents “read and write” (this is similar to the file permissions system in Linux, but is simplified for the sake of clarity); by checking the value of acs at line 10, the assertion can guarantee both admins approve the access and the permission type of is at least 1. ∎
Here the question we are interested is: when the assertion fails (referred as “Read Failure” in the following, i.e. the read access to fails to be granted), which entity (entities) in the program shall be responsible? The literature has several possible answers. By the definition of dependency ([1, 8, 42]), the value of acs depends on the value of apv and typ, which further depend on all three inputs. That is to say, the read failure depends on all variables in the program, thus program slicing techniques (both syntactic slicing [45] and semantic slicing [39]) would take the whole program as the slice related with read failure. Such a slice is useful in debugging in the sense that it rules out parts of the program that are completely irrelevant with the failure, and modifying any code left in the slice may prevent the failure, e.g. replacing “acs=apvtyp” with “acs=2” trivially fixes the read failure problem. However, this paper presumes the program code to be unmodifiable, hence a statement like “acs=apvtyp”, which is fully controlled by others and acts merely as the intermediary between causes and effects, shall not be treated as responsible. In addition, the third input (i.e. the system setting of ’s permission type) is also included in the slice. Although it does affect acs’s value, it is not decisive in this case (i.e. no matter it is 1 or 2, it could not either enforce or prevent the failure). Therefore, the dependency analysis and slicing are not precise enough for determining responsibility.
Causation by counterfactual dependency [31] examines the cause in every single execution and excludes non-decisive factors (e.g. the third input in this example), but it is too strong in some circumstances. For example, in an execution where both the first two inputs are 0, neither of them would be determined as the cause of read failure, because if one input is changed to value 1, the failure would still occur due to the other input 0.
Actual cause introduced in [21, 22] is based on the structural equations model (SEM) [11], and extends the basic notion of counterfactual dependency to allow “contingent dependency”. For this example here, it is straightforward to create a SEM to represent the access control program (although it is not always the case): three inputs are represented by exogenous variables, and five program variables are represented by endogenous variables, in which the value of apv is i1*i2. Consider an execution where both the first two inputs are 0, no matter what value the third input takes, the actual causes of read failure (i.e. acs<1) would be determined as “i1=0”, “i2=0”, “apv=0” and “acs=0”, since the failure counterfactually depends on each of them under certain contingencies. Thus, both two admins are equally determined as causes of failure, as well as two intermediary variables. This structural-model method has allowed for a great progress in causality analysis, and solved many problems of previous approaches. However, as an abstraction of concrete semantics, the SEM unnecessarily misses too much information, including the following three important points.
(P1) Time (i.e. the temporal ordering of events) should be taken into account. For example, the SEM does not keep the temporal ordering of first two inputs (i.e. the information that “i1=0” occurs before “i2=0” is missed), hence it determines both of them equally as the cause of assigning 0 to apv, further as the cause of read failure. However, in the actual execution where first two inputs are 0, the first input already decides the value of apv before the second input is entered and the assignment at line 7 is not even executed, thus it is unnecessary to take the second input as a cause of assigning 0 to apv or the read failure. To deal with this difficulty, Pearl’s solution is to modify the model and introduce new variables [7] to distinguish whether apv is assigned by i1 or i2. However, a much simpler method is to keep the temporal ordering of events, such that only the first event that ensures the behavior of interest is counted as the cause. Therefore, in an execution where both the first two inputs are 0, the first input ensures the read failure before the second input is entered, hence only the first input is responsible for failure; meanwhile, in another execution where the first input is 1 and the second one is 0, the second input is the first and only one that ensures the failure hence shall take the responsibility.
(P2) The cause must be free to make choices. For example, acs=0 is determined as an actual cause of read failure, based on the reasoning that if the endogenous variable acs in SEM is assigned a different value, say 2, then the read failure would not have occurred. But such a reasoning ignores a simple fact that acs is not free to choose its value and acts merely as an intermediary between causes and effects. Thus, only entities that are free to make choices can possibly be causes, and they include but are not limited to user inputs, system settings, files read, parameters of procedures or modules, returned values of external functions, variable initialization, random number generations and the parallelism. To be more accurate, it is the external subject (who does the input, configures the system settings, etc.) that is free to make choices, but we say that entities like user inputs are free to make choices, as an abuse of language.
(P3) It is necessary to specify “to whose cognizance / knowledge” when identifying the cause. All the above reasoning on causality is implicitly based on an omniscient observer’s cognizance (i.e. everything that occurred is known), yet it is non-trivial to consider the causality to the cognizance of a non-omniscient observer. Reconsider the access control program example, and suppose we adopt the cognizance of the second admin who is in charge of the second input. If she/he is aware that the first input is already 0, she/he would not be responsible for the failure; otherwise she/he does not know whether the first input is 0 or 1, then she/he is responsible for ensuring the occurrence of failure. In most cases, the cognizance of an omniscient observer will be adopted, but not always.
2.2 An Informal Definition of Responsibility
To take the above three points into account and build a more expressive framework, this paper proposes responsibility, whose informal definition is as follows.
Definition 1 (Responsibility, informally)
To the cognizance of an observer, the entity is responsible for a behavior of interest in a certain execution, if and only if, according to the observer’s observation, is free to choose its value, and such a choice is the first one that guarantees the occurrence of in that execution.
It is worth mentioning that, for the whole system whose semantics is a set of executions, there may exist more than one entities that are responsible for . Nevertheless, in every single execution where occurs, there is only one entity that is responsible for . To decide which entity in an execution is responsible, the execution alone is not sufficient, and it is necessary to reason on the whole semantics to exhibit the entity’s “free choice” and guarantee of . Thus, responsibility is not a trace property (neither safety nor liveness property).
To put such a definition into effect, our framework of responsibility analysis is designed as Fig. 2, which essentially consists of three components: (1) System semantics, i.e. the set of all possible executions, each of which can be analyzed individually. (2) A lattice of system behaviors of interest, which is ordered such that the stronger a behavior is, the lower is its position in the lattice. (3) An observation function for each observer, which maps every (probably unfinished) execution to a behavior in the lattice that is guaranteed to occur, even though such a behavior may have not occurred yet. These three components are formally defined in section 3, and their abstractions are sketched in the appendix 0.B.
In this framework, if an observer’s observation finds that the guaranteed behavior grows stronger after extending an execution, then the extension part of execution must be responsible for ensuring the occurrence of the stronger behavior. Consider the example in Fig. 2 which sketches the analysis for a certain execution of the access control program. Suppose in the lattice represents “not sure if the read access fails or not” and RF represents the behavior of read failure, whose formal definitions are given in section 3.2. The solid arrow from executions to the lattice stands for the observation of an omniscient observer, while the dashed arrow stands for the observation of the second admin who is unaware of the first input. As illustrated in the figure, the omniscient observer finds that the execution from point 1 to point 2 can guarantee only , while the stronger behavior RF is guaranteed if the execution reaches point 3. Thus, to the cognizance of the omniscient observer, “i1=0” between point 2 and 3 is responsible for the read failure. Meanwhile, the second admin observes that all the executions upto point 5 guarantee , and RF is guaranteed only after point 6 is reached. Hence, to the cognizance of the second admin, “i2=0” between point 5 and point 6 is responsible for the read failure. For the sake of completeness, the entire desired analysis result for Example 1 is included in the following.
Example 2 (Access Control, Continued)
To the cognizance of an omniscient observer: for any execution, if the first input is 0, no matter what the other two inputs are, only the first admin is responsible for the read failure; if the first input is 1 and the second one is 0, the second admin is responsible.
To the cognizance of the second admin, two cases need to be considered separately. If she/he is aware of the input of first admin, the analysis result is exactly the same as the omniscient observer. Otherwise, she/he does not know the first input: in every execution where the second input is 0, the second admin is responsible, no matter what the first and third input are; in every execution where the second input is 1, nobody shall be responsible for the failure, since whether the failure occurs or not is uncertain from the second admin’s perspective. ∎
After finishing responsibility analysis, it is time for the security admin to configure permissions granted to each responsible entity at her/his discretion. If the behavior of interest is desired or the responsible entity is authorized, the permissions granted to the responsible entity can be kept. On the contrary, if that behavior is undesired or it is against the policy for the responsible entity to control it, the permissions granted to the responsible entity shall be confined. For instance, in the access control program, if the first two inputs are from admins who are authorized to control the access, their permissions to input 0 and 1 can be kept; if those two inputs come from ordinary users who have no authorization to deny other users’ access, their permissions to input 0 shall be removed.
3 Formal Definition of Responsibility
In order to formalize the framework of responsibility analysis, this section introduces event traces to represent the system semantics, builds a lattice of system behaviors by trace properties, proposes an observation function that derives from the observer’s cognizance and an inquiry function on system behaviors. Furthermore, this section formally defines responsibility as an abstraction of system semantics, using the observation function. To strengthen the intuition of responsibility analysis, the analysis of Example 1 will be illustrated step by step.
3.1 System Semantics
Generally speaking, no matter what system we are concerned with and no matter which programming language is used to implement that system, the system’s semantics can be represented by event traces.
3.1.1 Event Trace
In general, an event could be used to represent any action in the system, such as “input an integer”, “assign a value to a variable”, or even “launch the program”. Take the classic While programming language as an example, there are only three types of events: skip, assignment, and Boolean test. In order to make the definition of responsibility as generic as possible, here we do not adopt a specific programming language or restrict the range of possible events.
A trace is a sequence of events that represents an execution of the system, and its length is the number of events in . If is infinite, then its length is denoted as . A special trace is the empty trace , whose length is 0. A trace is - less than or equal to another trace , if and only if, is a prefix of . The concatenation of a finite trace and an event e is simply defined by juxtaposition , and the concatenation of a finite traces and another (finite or infinite) trace is denoted as .
[TABLE]
The function returns the prefixes of every trace in the set of traces.
[TABLE]
3.1.2 Trace Semantics
For any system that we are concerned with, its maximal trace semantics, denoted as , is the set of all possible maximal traces of that system. Especially, the maximal trace semantics of an empty program is . Correspondingly, the prefix trace semantics is the set of all possible prefix traces, which is an abstraction of maximal trace semantics via , i.e. . Besides, a trace is said to be valid in the system, if and only if . Obviously, both maximal and prefix trace semantics do preserve the temporal ordering of events, which is missed by the SEM.
Example 3 (Access Control, Continued)
For the program in Fig. 1, only two types of events are used: assignment (e.g. apv=1) and Boolean test (e.g. i1==0 and , where denotes the failure of a Boolean test). To clarify the boundary among events, the triangle is used in the following to separate events in the trace. The access control program has three inputs, each of which has two possible values, thus its maximal trace semantics consists of 8 traces (T1-, each of which is represented as a path in Fig. 3 starting at the entry point of program and finishing at the exit point. E.g. apv=1 i1=0 i1==0 apv=0 i2=0 typ=1 acs=0 denotes the maximal execution where the first two inputs are 0 and the third input is 1. Meanwhile, the prefix trace semantics are represented by the paths that start at the entry point and stop at any point (including the entry point for the empty trace ). ∎
3.2 Lattice of System Behaviors of Interest
3.2.1 Trace Property
A trace property is a set of traces in which a given property holds. Most behaviors of a given system, if not all, can be represented as a maximal trace property .
Example 4 (Access Control, Continued)
As illustrated in Fig. 3, the behavior “Read Failure” RF is represented as a set of maximal traces such that the last event is , i.e. RF = { = } = {T1, T2, T3, T4, T5, T6}; the behavior “Read Success” RS (i.e. the read access succeeds to be granted) is the complement of RF, i.e. RS = = {T7, T8}, whose subset RO = {T7} and RW = {T8} represent stronger properties “Read Only access is granted” and “Read and Write access is granted”, respectively. ∎
3.2.2 Complete Lattice of Maximal Trace Properties of Interest
We build a complete lattice of maximal trace properties, each of which represents a behavior of interest. Typically, such a lattice is of form \langle\mathcal{L}^{\mathsf{Max}},\allowbreak\>\subseteq,\allowbreak\>\top^{\mathsf{Max}},\allowbreak\>\bot^{\mathsf{Max}},\allowbreak\>\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup},\allowbreak\>\mathop{\hbox to0.0pt{\Large\mskip 1.6mu\cdot\hss}\cap}\rangle, where
- –
is a set of behaviors of interest, each of which is represented by a maximal trace property;
- –
, i.e. the top is the weakest maximal trace property which holds in every valid maximal trace;
- –
, i.e. the bottom is the strongest property such that no valid trace has this property, hence it is used to represent the property of invalidity;
- –
is the standard set inclusion operation;
- –
\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup} and \mathop{\hbox to0.0pt{\Large\mskip 1.6mu\cdot\hss}\cap} are join and meet operations, which might not be the standard and , since is a subset of but not necessarily a sublattice.
For any given system, there is possibly more than one way to build the complete lattice of maximal trace properties, depending on which behaviors are of interest. A special case of lattice is the power set of maximal trace semantics, i.e. , which can be used to examine the responsibility for every possible behavior in the system. However, in most cases, a single behavior is of interest, and it is sufficient to adopt a lattice with only four elements: representing the behavior of interest, representing the complement of the behavior of interest, as well as the top and bottom . Particularly, if is equal to , i.e. every valid maximal trace in the system has this behavior of interest, then a trivial lattice with only the top and bottom is built, from which no responsibility can be found, making the corresponding analysis futile.
Example 5 (Access Control, Continued)
We assume that “Read Failure” is of interest, as well as the behavior of granting write access. As illustrated by the lattice in Fig. 2, regarding whether the read access fails or not, the top is split into two properties “Read Failure” RF and “Read Success” RS, which are defined in Example 4 such that RF \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup} RS = and RF \mathop{\hbox to0.0pt{\Large\mskip 1.6mu\cdot\hss}\cap} RS = . Furthermore, regarding whether the write access is granted or not, RS is split into “Read Only access is granted” RO and “Read and Write access is granted” RW. Now every property of interest corresponds to an element in the lattice, and the bottom = is the meet \mathop{\hbox to0.0pt{\Large\mskip 1.6mu\cdot\hss}\cap} of RF, RO and RW. In addition, if “Read Failure” is the only behavior of interest, RO and RW can be removed from the lattice. ∎
3.2.3 Prediction Abstraction
Although the maximal trace property is well-suited to represent system behaviors, it does not reveal the point along the maximal trace from which a property is guaranteed to hold later in the execution. Thus, we propose to abstract every maximal trace property isomorphically into a set of prefixes of maximal traces in , excluding those whose maximal prolongation may not satisfy the property . This abstraction is called prediction abstraction, and is a prediction trace property corresponding to . It is easy to see that is a superset of , and is not necessarily prefix-closed.
[TABLE]
We have a Galois isomorphism between maximal trace properties and prediction trace properties:
[TABLE]
where the abstract domain is obtained by a function , which is defined as . The following lemma immediately follows from the definition of .
Lemma 1
Given a prediction trace property that corresponds to a maximal trace property , if a prefix trace belongs to , then guarantees the satisfaction of property (i.e. every valid maximal trace that is greater than or equal to is guaranteed to have property ).
Example 6 (Access Control, Continued)
By , each behavior in the lattice of Example 5 can be abstracted into a prediction trace property:
- –
, i.e. every valid trace in guarantees .
- –
apv=1 i1=0 apv=1 i1=1 i2=0 , i.e. for any valid trace, if at least one of first two inputs is 0, then it guarantees “Read Failure” RF.
- –
apv=1 i1=1 i2=1 , i.e. for any valid trace, if first two inputs are 1, it guarantees “Read Success” RS.
- –
apv=1 i1=1 i2=1 typ=1 , i.e. for any valid trace, if first two inputs are 1 and the third input is 1, then it guarantees “Read Only access is granted” RO.
- –
apv=1 i1=1 i2=1 typ=2 , i.e. for any valid trace, if first two inputs are 1 and the third is 2, then it guarantees “Read and Write access is granted” RW.
- –
, i.e. no valid trace can guarantee . ∎
3.3 Observation of System Behaviors
Let be the maximal trace semantics and be the lattice of system behaviors designed as in Section 3.2. Given any prefix trace , an observer can learn some information from it, more precisely, a maximal trace property that is guaranteed by from the observer’s perspective. In this section, an observation function is proposed to represent such a “property learning process” of the observer, which is formally defined in the following three steps.
3.3.1 Inquiry Function
First, an inquiry function is defined to map every trace to the strongest maximal trace property in that can guarantee.
[TABLE]
Specially, for an invalid trace , there does not exist any such that , therefore .
Corollary 1
Given the semantics and lattice of system behaviors, if the inquiry function maps a trace to a maximal trace property , then guarantees the satisfaction of (i.e. every valid maximal trace that is greater than or equal to is guaranteed to have property ).
Lemma 2
The inquiry function is decreasing on the inquired trace : the greater (longer) is, the stronger property it can guarantee.
Example 7 (Access Control, Continued)
Using defined in Example 3 and defined in Example 5, the inquiry function of definition (3.3.1) is such that:
- –
= , i.e. apv=1 can guarantee only .
- –
= RF, i.e. after setting the first input as 0, “Read Failure” RF is guaranteed.
- –
, i.e. if the first input is 1, only is guaranteed before entering the second input.
- –
, i.e. if the second input is 0, “Read Failure” RF is guaranteed.
- –
, i.e. if first two inputs are 1, “Read Success” RS is guaranteed.
- –
i1=1 i2=1 = RW, i.e. if first two inputs are 1, after the third input is set to be 2, a stronger property “Read and Write access is granted” RW is guaranteed. ∎
3.3.2 Cognizance Function
As discussed in (P3) of section 2.1, it is necessary to take the observer’s cognizance into account. Specifically, in program security, the observer’s cognizance can be used to represent attackers’ capabilities (e.g. what they can learn from the program execution). Given a trace (not necessarily valid), if the observer cannot distinguish from some other traces, then he does not have an omniscient cognizance of , and the cognizance function is defined to include all traces indistinguishable from .
[TABLE]
Such a cognizance function is extensive, i.e. . In particular, there is an omniscient observer and its corresponding cognizance function is denoted as such that , which means that every trace is unambiguous to the omniscient observer.
To facilitate the proof of some desired properties for the observation function defined later, two assumptions are made here without loss of generality:
- (A1)
The cognizance of a trace is the concatenation of cognizances of and . I.e. = . 2. (A2)
Given an invalid trace, the cognizance function would not return a valid trace. I.e. = .
To make the assumption (A1) sound, we must have , because otherwise, for any non-empty trace , = does not have a fixpoint. In practice, is an equivalence relation, but the symmetry and transitivity property are not used in the proofs.
Example 8 (Access Control, Continued)
Consider two separate observers.
(i) For an omniscient observer: .
(ii) For an observer representing the second admin who is unaware of the first input: {i1=0 i1==0 apv=0, i1=1 }, i.e. this observer cannot distinguish whether the first input is 0 or 1. Thus, for a prefix trace in which the first two inputs are 0, = {apv=1 i1=0 i1==0 apv=0 i2=0, apv=1 i1=1 i2=0}, where apv=1 and i2=0 are known by this observer. In the same way, its cognizance on other traces can be generated. ∎
3.3.3 Observation Function
For an observer with cognizance function , given a single trace , the observer cannot distinguish with traces in . In order to formalize the information that the observer can learn from , we apply the inquiry function on each trace in , and get a set of maximal trace properties. By joining them together, we get the strongest property in that can guarantee from the observer’s perspective. Such a process is defined as the observation function .
[TABLE]
From the above definition, it is easy to see that, for every invalid trace , , since every trace in is invalid by (A2) and . In addition, for an omniscient observer with cognizance function , its observation .
Corollary 2
For any observer with cognizance , if the corresponding observation function maps a trace to a maximal trace property , then guarantees the satisfaction of property (i.e. every valid maximal trace that is greater than or equal to is guaranteed to have property ).
Lemma 3
The observation function is decreasing on the observed trace : the greater (longer) is, the stronger property it can observe.
Example 9 (Access Control, Continued)
For an omniscient observer, the observation function is identical to the inquire function in Example 7. If the cognizance of the second admin defined in Example 8 is adopted, we get an observation function that works exactly the same as the dashed arrows in Fig.2:
- –
\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup} RF \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup} = , i.e. even if the first input is already 0 in the trace, no property except can be guaranteed for the second admin.
- –
\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup} RF \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup} , i.e. if the second input is 1, only can be guaranteed.
- –
\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup} \texttt{RF}\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\texttt{RF} , i.e. RF is guaranteed only after the second input is entered 0. ∎
3.4 Formal Definition of Responsibility
Using the three components of responsibility analysis introduced above, responsibility is formally defined as the responsibility abstraction in (3.4). Specifically, the first parameter is the maximal trace semantics , the second parameter is the lattice of system behaviors, the third parameter is the cognizance function of a given observer, the fourth parameter is the behavior whose responsibility is of interest, and the last parameter is the analyzed traces .
Consider every trace where H, R and F respectively stand for History, Responsible part and Future. If holds, then does not guarantee the behavior , while guarantees a behavior which is at least as strong as and is not the invalidity property represented by . Therefore, is said to be responsible for ensuring behavior in the trace .
In particular, the length of is restricted to be 1 (i.e. ), such that the responsible entity must be a single event and the responsibility analysis could be as refined as possible. Otherwise, if we do not have such a restriction, then for every analyzed trace where the behavior holds, the responsibility analysis may split the trace into three parts such that , and . In such a case, holds, and the whole trace would be found responsible for . This result is trivially correct, but too coarse to be useful in practice.
\halign to=0.0pt{\@initline\setbox\@itembox\hbox\bgroup\copy@itemleftstrutbox\displaystyle{#}\csname r@fil0\endcsname@lastsep&\llap{#}\cr}\hfil\ignorespaces&\hbox to0.0pt{\hss@arstrut(16)\crcr\penalty 2500\cr}\hfil\ignorespaces&\hbox to0.0pt{\hss@arstrut\hbox{(\ref{eq:alpharesponsibility}){}{}}\crcr\penalty 2500\cr}\hfil\ignorespaces&\hbox to0.0pt{\hss@arstrut(18)\crcr\penalty 2500\cr}\hfil\ignorespaces&\hbox to0.0pt{\hss@arstrut(19)\crcr\penalty 2500\cr\hbox{\vrule height=7.5pt,depth=0.0pt,width=0.0pt}\displaystyle{\quad let\;\mathbb{I}({\mathcal{S},\mathcal{L},\sigma})=\mathop{\hbox to0.0pt{\Large\mskip 1.6mu\cdot\hss}\cap}\{\mathcal{P}\in\mathcal{L}\mid\sigma\in\alpha_{\mathsf{Pred}}\llbracket\mathcal{S}\rrbracket({\mathcal{P}})\}\;in{}}\hbox{\vrule height=0.0pt,depth=3.5pt,width=0.0pt}}\hfil\ignorespaces&\hbox to0.0pt{\hss@arstrut(20)\crcr\penalty 2500\cr\hbox{\vrule height=7.5pt,depth=0.0pt,width=0.0pt}\displaystyle{\quad\quad let\;\mathbb{O}({\mathcal{S},\mathcal{L},\mathbb{C},\sigma})=\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S},\mathcal{L},\sigma^{\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\}\;in{}}\hbox{\vrule height=0.0pt,depth=3.5pt,width=0.0pt}}\hfil\ignorespaces&\hbox to0.0pt{\hss@arstrut(21)\crcr\penalty 2500\cr}\hfil\ignorespaces&\hbox to0.0pt{\hss@arstrut(22)\crcr\penalty 2500\cr}\hfil\ignorespaces&\hbox to0.0pt{\hss@arstrut(23)\crcr}$
Since preserves joins on analyzed traces , we have a Galois connection: \langle\wp(\textrm{E}^{\ast\infty}),\allowbreak\>\subseteq\rangle\mathrel{\hbox to0.0pt{\smash{\hskip 0.0pt\raisebox{-3.57222pt}{\hbox to81.80313pt{\smash{-}\mkern-7.0mu\cleaders\hbox{}\hfill\mkern-7.0mu\mathord{\rightarrow}}}}\hss}\hbox to0.0pt{\smash{\hskip 0.0pt\raisebox{-0.01561pt}{\hbox to81.80313pt{\mathord{\leftarrow}\mkern-7.0mu\cleaders\hbox{}\hfill\mkern-7.0mu\smash{-}}}}\hss}\hbox to0.0pt{\smash{\hskip 7.07304pt\raisebox{-7.8722pt}{\ \scriptstyle\alpha_{R}({\mathcal{S}^{\mathsf{Max}},,\mathcal{L}^{\mathsf{Max}},,\mathbb{C},,\mathcal{B}})\ }}\hss}\hbox to0.0pt{\smash{\hskip 5.0pt\raisebox{7.88995pt}{\ \scriptstyle\gamma_{R}(\mathcal{S}^{\mathsf{Max}},,\mathcal{L}^{\mathsf{Max}},,\mathbb{C},,\mathcal{B})\ }}\hss}\rule[-9.6222pt]{0.0pt}{23.23157pt}\hskip 81.80313pt}\langle\wp(\textrm{E}^{\ast\infty}\!\times\textrm{E}\times\textrm{E}^{\ast\infty}),\allowbreak\>\subseteq\rangle.
Lemma 4
If is said to be responsible for a behavior in a valid trace , then guarantees the occurrence of behavior , and there must exist another valid prefix trace such that the behavior is not guaranteed.
Recall the three desired points (time, free choices and cognizance) for defining responsibility in section 2.1. It is obvious that has taken both the temporal ordering of events and the observer’s cognizance into account. As for the free choices, it is easy to find from lemma 4 that, if is determined by its history trace and is not free to make choices (i.e. ), then cannot be responsible for any behavior in the trace .
3.5 Responsibility Analysis
To sum up, the responsibility analysis typically consists of four steps: I) collect the system’s trace semantics (in Section 3.1.2); II) build the complete lattice of maximal trace properties of interest (in Section 3.2.2); III) derive an inquiry function from , define a cognizance function for each observer, and create the corresponding observation function (in Section 3.3.3); IV) specify the behavior of interest and the analyzed traces , and apply the responsibility abstraction to get the analysis result (in Section 3.4). Hence, the responsibility analysis is essentially an abstract interpretation of the event trace semantics.
In the above definition of responsibility, the semantics and lattice of system behaviors are concrete, and they are explicitly displayed in the access control example for the sake of clarity. However, they may be uncomputable in practice, and we do not require programmers to provide them in the implementation of responsibility analysis. Instead, they are provided in the abstract, using an abstract interpretation-based static analysis that is sketched in the appendix 0.B.
Example 10 (Access Control, Continued)
Using the observation functions created in example 9, the abstraction can analyze the responsibility of a certain behavior in the set of traces. Suppose we want to analyze “Read Failure” in every possible execution, then is RF, and includes all valid maximal traces, i.e. . Thus, computes the responsibility analysis result, which is essential the same as desired in Example 2.
Furthermore, the responsibility of “granting write access” can be analyzed by setting the behavior as RW instead, and we get the following result. To the cognizance of an omniscient observer, in every execution that both the first two inputs are 1, the third input (i.e. system setting of permission type) is responsible for RW. Meanwhile, to the cognizance of the second admin who is unaware of the first input, no one is found responsible for RW, because whether the write access fails or not is always uncertain, from the second admin’s perspective. ∎
4 Examples of Responsibility Analysis
Responsibility is a broad concept, and our definition of responsibility based on the abstraction of event trace semantics is universally applicable in various scientific fields. We have examined every example supplied in actual cause [21, 22] and found that our definition of responsibility can handle them well, in which actions like “drop a lit match in the forest” or “throw a rock at the bottle” are treated as events in the trace. In the following, we will illustrate the responsibility analysis by two more examples: the “negative balance” problem of a withdrawal transaction, and the information leakage problem.
4.1 Responsibility Analysis of “Negative Balance” Problem
Example 11 (Negative Balance)
Consider the withdrawal transaction program in Fig. 5 in which the query_database() function gets the balance of a certain bank account before the transaction, and input() specifies the withdrawal amount that is positive. When the withdrawal transaction completes, if the balance is negative, which entity in the program shall be responsible for it? ∎
It is not hard to see that, the “negative balance” problem can be transformed into an equivalent buffer overflow problem, where the memory of size balance is allocated and the index at n-1 is visited. Although this problem has been well studied, it suffices to demonstrate the advantages of responsibility analysis over dependency/causality analysis.
As discussed in section 3.5, the responsibility analysis consists of four steps. For the sake of simplicity, we consider only the omniscient observer here.
- (1)
Taking each assignment as an event, each maximal trace in this program is of length 3, and the program’s maximal trace semantics consists of infinite number of such traces. E.g. balance=0 n=5 balance=-5 denotes a maximal execution, in which the balance before the transaction is 0 and the withdrawal amount is 5 such that “negative balance” occurs. 2. (2)
Since “negative balance” is the only behavior that we are interested here, a lattice of maximal trace properties in Fig. 5 with four elements can be built, where NB (Negative Balance) is the set of maximal traces where the value of balance is negative at the end, and is its complement. 3. (3)
Using the omniscient observer’s cognizance , the observation function can be easily derived from the lattice , such that:
- –
;
- –
where , i.e. if the balance before the transaction is negative or 0, the occurrence of “negative balance” is guaranteed before the withdrawal amount n is entered;
- –
where , i.e. if the balance before the transaction is strictly greater than 0, whether “negative balance” occurs or not has not been decided;
- –
where and , i.e. “negative balance” is guaranteed to occur immediately after input() returns a value strictly greater than balance;
- –
where and , i.e. “negative balance” is guaranteed not to occur immediately after input() returns a value less than or equal to balance. 4. (4)
Suppose the behavior and the analyzed traces , the abstraction gets the following result. If query_database() returns 0 or a negative value, no matter what value input() returns, the function query_database() (i.e. event balance=i) is responsible for “negative balance”, and further responsibility analysis shall be applied on the previous transactions of the database. Otherwise, if query_database() returns a value strictly greater than 0, the function input() (i.e. event n=j) takes the responsibility for “negative balance”, thus “negative balance” can be prevented by configuring the permission granted to input() such that its permitted return value must be less than or equal to the returned value of query_database().
4.2 Responsibility Analysis of Information Leakage
Essentially, responsibility analysis of information leakage is the same as read failure or “negative balance” problem, and the only significant distinction is on defining the behaviors of interest. Here we adopt the notion of non-interference [17] to represent the behavior of information leakage.
In the program, the inputs and outputs are classified as either Low (public, low sensitivity) or High (private, high sensitivity). For a given trace , if there is another trace such that they have the same low inputs but different low outputs, then the trace is said to leak private information. If no trace in the program leaks private information (i.e. every two traces with the same low inputs have the same low outputs, regardless of the high inputs), the program is secure and has the non-interference property. Thus, for any program with maximal trace semantics , the behavior of “Information Leakage” IL is represented as the set of leaky traces, i.e. IL , where functions and collects low inputs and outputs along the trace, respectively. The behavior of “No information Leakage” NL is the complement of IL, i.e. NL . Thus, the lattice of maximal trace properties regarding information leakage can be built as in in Fig. 6. Further, the corresponding observation function can be created, and the analysis result can be obtained by applying the responsibility abstraction.
Notice that we are interested in analyzing only the insecure programs in which some traces leak private information while others do not, i.e. . For the erroneous programs where every trace leaks private information, i.e. , we need to admit that our responsibility analysis cannot identify any entity responsible for the leakage, unless “launching the program” is treated as an event and it would be found responsible for leaking private information.
5 Related Work
5.0.1 Definition of Causality and Responsibility
Hume [25] is the first one to specify causation by counterfactual dependence [32]. The best known counterfactual theory of causation is proposed by Lewis [31], which defines causation as a transitive closure of counterfactual dependencies. Halpern and Pearl [21, 22, 33] defines actual causality based on SEM and extends counterfactual dependency to allow “contingent dependency”. Chockler and Halpern [9] defines responsibility to have a quantitative measure of the relevance between causes and effects, and defines blame to consider the epistemic state of an agent. Their application of actual causality, responsibility and blame is mainly on artificial intelligence.
Our definition of responsibility also adopts the idea of counterfactual dependence in the sense that, suppose an event is said to be responsible for behavior in the trace , there must exist another event such that, if is replaced by , then is not guaranteed (by lemma 4).
5.0.2 Error Cause Localization
Classic program analysis techniques, e.g. dependency analysis [1, 8, 42] and program slicing [45, 44, 30, 2], are useful in detecting the code that may be relevant to errors, but fail to localize the cause of error.
In recent years, there are many papers [4, 20, 28, 19, 38, 37, 36, 27] on fault localization for counterexample traces, and most of them compare multiple traces produced by a model checker and build a heuristic metric to localize the point from which error traces separate from correct traces. Other related papers include error diagnosis by abductive/backward inference [15], tracking down bugs by dynamic invariant detection [23]. Actual causality is applied to explain counterexamples from model checker [6] and estimate the coverage of specification [10]. Besides, there are researches on analyzing causes of specific security issues. E.g. King et al. [29] employ a blame dependency graph to explain the source of information flow violation and generate a program slice as the error report.
Compared to the above techniques, this paper succeeds to formally define the cause or responsibility, and the proposed responsibility analysis, which does not require a counterexample from the model checker, is sound, scalable and generic to cope with various problems.
6 Conclusion and Future Work
This paper formally defines responsibility as an abstraction of event trace semantics. Typically, the responsibility analysis consists of four steps: collect the trace semantics, build a lattice of behaviors of interest, create an observation function for each observer, and apply the responsibility abstraction on analyzed traces. Its effectiveness has been demonstrated by several examples.
In the future, we intent to: (1) formalize the abstract responsibility analysis that is sketched in the appendix 0.B, (2) build a lattice of responsibility abstractions to cope with possible alternative weaker or stronger definitions of responsibility, (3) generalize the definition of cognizance function as an abstraction of system semantics, and (4) study the responsibility analysis of probabilistic programs.
Acknowledgment
This work was supported in part by NSF Grant CNS-1446511. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. P. Cousot thanks Marco Pistoia for initial discussions on responsibility while visiting the Thomas J. Watson Research Center at Hawthorne in 2005.
Appendix 0.A Appended Proofs
0.A.1 Proof of Galois Isomorphism (3.2.3)
[TABLE]
Proof
First, we prove that and are increasing.
@firsttab@firsttab@firsttab@@startline
= = >
\ >
def. \subseteq$$\Rbag
\ >
def. \vee$$\Rbag
\ >
def. \subseteq$$\Rbag
\ >
\ >
def. \Rightarrow$$\Rbag
\ >
\ >
def. and is increasing
\ >
def. \cap$$\Rbag
\ >
def. \alpha_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket$$\Rbag
@stopline
@firsttab@firsttab@firsttab@@startline
= = >
\ >
def. \cap$$\Rbag
\ >
def. \gamma_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket$$\Rbag
@stopline
Then, we prove that and are identity functions.
@firsttab@firsttab@firsttab@@startline
= = >
\
>
\ >
def. \alpha_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket$$\Rbag
\
>
\>
\Lbag$$\mathcal{P}=\mathsf{Pref}(\mathcal{P})\cap\mathcal{S}^{\mathsf{Max}} since \mathcal{P}\in\wp(\mathcal{S}^{\mathsf{Max}})$$\Rbag
\
>
\ >
def. \gamma_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket$$\Rbag
\
>
\Lbag$$\mathcal{S}^{\mathsf{Max}}\cap(\mathsf{Pref}(\mathcal{P})\backslash\mathcal{S}^{\mathsf{Max}})=\emptyset$$\Rbag
\
>
\Lbag$$\mathcal{P}\in\wp(\mathcal{S}^{\mathsf{Max}})$$\Rbag
@stopline
@firsttab@firsttab@firsttab@@startline
= = >
\
>
\ >
\Lbag$$\mathcal{Q}\in\bar{\alpha}_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket({\wp(\mathcal{S}^{\mathsf{Max}})}), thus \mathcal{Q}=\alpha_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket({\mathcal{P}^{\prime}})$$\Rbag
\
>
\Lbag$$\gamma_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket\circ\alpha_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket({\mathcal{P}^{\prime}})=\mathcal{P}^{\prime}$$\Rbag
\
>
by the assumption \mathcal{Q}=\alpha_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket({\mathcal{P}^{\prime}})$$\Rbag
@stopline
By the above four properties, and form a Galois isomorphism.
0.A.2 Proofs for Lemma 2
Lemma 5
Given the semantics and lattice of system behaviors, for any maximal trace property , if a trace belongs to the prediction trace property that corresponds to , then every valid trace greater than belongs to the prediction trace property too. I.e. .
Proof
Proof by contradiction. Here we assume . By the definition of prediction abstraction, . There are two possibilities for :
- , hence every maximal trace greater than does not belong to ; or 2) . Both cases imply that there is a maximal trace such that , which contradicts with the assumption of .
Lemma 2. Given the semantics and lattice of system behaviors, the corresponding inquiry function is decreasing on the inquired trace : the greater is, the stronger property it can guarantee. I.e. .
Proof
First, if is invalid (i.e. ), then every trace that is greater than must also be invalid (i.e. ), hence .
Second, if , then , hence .
Last, if , then
@firsttab@firsttab@firsttab@@startline
= =>
\ >
lemma. 5
\ >
\ >
def. \Rightarrow$$\Rbag
\ >
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.6mu\cdot\hss}\cap}\{\mathcal{P}\in\mathcal{L}^{\mathsf{Max}}\mid\sigma\in\alpha_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket({\mathcal{P}})\}\supseteq\mathop{\hbox to0.0pt{\Large\mskip 1.6mu\cdot\hss}\cap}\{\mathcal{P}\in\mathcal{L}^{\mathsf{Max}}\mid\sigma^{\prime}\in\alpha_{\mathsf{Pred}}\llbracket\mathcal{S}^{\mathsf{Max}}\rrbracket({\mathcal{P}})\}
\ >
def. \mathop{\hbox to0.0pt{\Large\mskip 1.6mu\cdot\hss}\cap}$$\Rbag
\ >
def. \mathbb{I}$$\Rbag
@stopline
0.A.3 Proof of Corollary 2
Corollary 2. Given the semantics and lattice of system behaviors, for any observer with cognizance , if the observation function maps a trace to a maximal trace property , then guarantees the satisfaction of property (i.e. every valid maximal trace that is greater than or equal to is guaranteed to have property ).
Proof
Suppose . By the corollary 1, guarantees the satisfaction of property , i.e. every valid maximal trace that is greater than or equal to belongs to .
In addition, by the definition of observation function in (3.3.3), we know =\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\} , since the cognizance is extensive (i.e. ). Therefore, every valid maximal trace that is greater than or equal to belongs to . That is to say, guarantees the satisfaction of property .
0.A.4 Proof of Lemma 3
Lemma 3. Given the semantics , lattice of system behaviors and cognizance function , the observation function is decreasing: the longer the observed trace is, the stronger property it can observe. I.e. .
Proof
First, if is invalid (i.e. ), then every trace that is greater than must also be invalid (i.e. ), hence it is easy to find that .
Second, if , then we have . Hence, it is trivial to find .
Last, if , then must be a valid non-maximal trace, i.e. . From Lemma 7, it is easy to see . Since is a prolongation of with events, by the transitivity of , we can prove that .
0.A.5 Proofs for Lemma 4
Lemma 6
Given the semantics and the lattice of system behaviors, for any maximal trace property and any valid prefix trace that is not maximal, if every valid prefix trace which concatenates with a new event e belongs to the prediction trace property , then belongs to too. I.e. .
Proof
Prove by contradiction. Assume that . By the definition that . In order to have , there are two possibilities.
-
. This implies that , which further implies that . Since , there must exist at least one e such that .
-
There is a maximal trace such that . Take , then holds, which implies .
Both two cases find that , which contradicts with the assumption .
Corollary 3
For any cognizance , we have \underset{\textrm{{e}}\in\textrm{E}}{\mathop{\hbox to0.0pt{\raisebox{-1.29167pt}[0.0pt][0.0pt]{\LARGE\mskip 3.5mu\cdot}\hss}\bigcup}\limits}\mathbb{C}({\textrm{{e}}})\supseteq\textrm{E}.
Proof
This corollary comes immediately from the fact that the cognizance function is extensive.
Corollary 4
Given the semantics and lattice of behaviors, \mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma})=\underset{\textrm{{e}}\in\textrm{E}}{\mathop{\hbox to0.0pt{\raisebox{-1.29167pt}[0.0pt][0.0pt]{\LARGE\mskip 3.5mu\cdot}\hss}\bigcup}\limits}\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})=\underset{\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}}{\mathop{\hbox to0.0pt{\raisebox{-1.29167pt}[0.0pt][0.0pt]{\LARGE\mskip 3.5mu\cdot}\hss}\bigcup}\limits}\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}}).
Proof
First, \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\textrm{{e}}\in\textrm{E}\} =(\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}\}) \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}(\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\sigma\textrm{{e}}\not\in\mathcal{S}^{\mathsf{Pref}}\}) =(\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}\})\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\bot^{\mathsf{Max}} =\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}\}.
Second, we prove \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}\} in two steps:
- by lemma. 2, it is proved that , thus \mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma})\supseteq\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}\}.
- assume \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}\} . By the definition of in (3.3.1), we know that and , which is impossible by lemma. 6. Thus, by contradiction, \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma\textrm{{e}}})\mid\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}\}.
Lemma 7
Given the semantics , the lattice of system behaviors and the cognizance function , we have:
* \ =\underset{\textrm{{e}}\in\textrm{E}}{\mathop{\hbox to0.0pt{\raisebox{-1.29167pt}[0.0pt][0.0pt]{\LARGE\mskip 3.5mu\cdot}\hss}\bigcup}\limits}\mathbb{O}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\mathbb{C},\sigma\textrm{{e}}})=\underset{\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}}{\mathop{\hbox to0.0pt{\raisebox{-1.29167pt}[0.0pt][0.0pt]{\LARGE\mskip 3.5mu\cdot}\hss}\bigcup}\limits}\mathbb{O}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\mathbb{C},\sigma\textrm{{e}}}).*
Proof
We start from the right side.
@firsttab@firsttab@firsttab@@startline
= =>
\displaystyle\underset{\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}}{\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}}\mathbb{O}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\mathbb{C},\sigma\textrm{{e}}})
\
>
\displaystyle(\underset{\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}}{\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}}\mathbb{O}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\mathbb{C},\sigma\textrm{{e}}}))\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\bot^{\mathsf{Max}} def.\bot^{\mathsf{Max}}$$\Rbag
\
>
\displaystyle\underset{\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}}{\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}}\mathbb{O}({\mathcal{S}^{\mathsf{Max}}\!,\mathcal{L}^{\mathsf{Max}}\!,\mathbb{C},\sigma\textrm{{e}}})\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\underset{\sigma\textrm{{e}}\not\in\mathcal{S}^{\mathsf{Pref}}}{\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}}\mathbb{O}({\mathcal{S}^{\mathsf{Max}}\!,\mathcal{L}^{\mathsf{Max}}\!,\mathbb{C},\sigma\textrm{{e}}})
\
>
\displaystyle\underset{\textrm{{e}}\in\textrm{E}}{\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}}\mathbb{O}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\mathbb{C},\sigma\textrm{{e}}}) merge two cases
\
>
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\tau})\mid\tau\in\mathbb{C}({\sigma\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\} def. \mathbb{O}$$\Rbag
\
>
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\mid\sigma^{\prime}\sigma^{\prime\prime}\in\mathbb{C}({\sigma\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\} replace with \sigma^{\prime}\sigma^{\prime\prime}$$\Rbag
\
>
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\mathbb{C}({\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\} assumption (A1)
\
>
\displaystyle(\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\mathbb{C}({\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\wedge|\sigma^{\prime\prime}|=1\})\;\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\;(\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\mathbb{C}({\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\wedge|\sigma^{\prime\prime}|=0\})\;\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\;(\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\mathbb{C}({\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\wedge|\sigma^{\prime\prime}|>1\})
\ >
split into 3 cases by the length of \sigma^{\prime\prime}$$\Rbag
@stopline
The first case:
@firsttab@firsttab@firsttab@@startline
= =>
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\mathbb{C}({\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\wedge|\sigma^{\prime\prime}|=1\}
\
>
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\textrm{E}\} corol. 3
\
>
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\} corollary 4
\
>
def. \mathbb{O}$$\Rbag
@stopline
The second case: if there exists s.t. , then \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}}) \mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\mathbb{C}({\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\wedge|\sigma^{\prime\prime}|=0\}=\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\}=\mathbb{O}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\mathbb{C},\sigma}). Otherwise, it is an empty set.
The third case:
@firsttab@firsttab@firsttab@@startline
= =>
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\mathbb{C}({\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\wedge|\sigma^{\prime\prime}|>1\}
\ >
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\wedge\sigma^{\prime\prime}\in\mathbb{C}({\textrm{{e}}})\wedge\textrm{{e}}\in\textrm{E}\wedge|\sigma^{\prime\prime}|>1\}
\ >
\Lbag$$\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}\sigma^{\prime\prime}})\subseteq\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}})$$\Rbag
\ >
\displaystyle\mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}\{\mathbb{I}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\sigma^{\prime}})\mid\sigma^{\prime}\in\mathbb{C}({\sigma})\} def. \mathop{\hbox to0.0pt{\Large\mskip 1.7mu\cdot\hss}\cup}$$\Rbag
\
>
def. \mathbb{O}$$\Rbag
@stopline
Joining the above three cases together, we have proved that \ \underset{\sigma\textrm{{e}}\in\mathcal{S}^{\mathsf{Pref}}}{\mathop{\hbox to0.0pt{\raisebox{-1.29167pt}[0.0pt][0.0pt]{\LARGE\mskip 3.5mu\cdot}\hss}\bigcup}\limits}\mathbb{O}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\mathbb{C},\sigma\textrm{{e}}})=\mathbb{O}({\mathcal{S}^{\mathsf{Max}},\mathcal{L}^{\mathsf{Max}},\mathbb{C},\sigma}).
Lemma 4. If is said to be responsible for a behavior in a valid trace , then guarantees the occurrence of behavior , and there must exist another valid prefix trace such that the behavior is not guaranteed.
Proof
First, from the definition of responsiblity, we know . By corollary 2, guarantees the satisfaction of , which is at least as strong as . Thus, the occurrence of behavior is guaranteed.
Second, we prove by contradiction. Assume that every valid trace guarantees the occurrence of behavior (i.e. ). By lemma 7, we can prove that , which contradicts with the requirement term of for to be responsible for the behavior .
Appendix 0.B Abstract Responsibility Analysis
In general the concrete maximal trace semantics is not computable, thus the static responsibility analysis proposed in Section 3 is undecidable, and an implementation of it has to abstract sets of finite or infinite traces involved in , , , , and .
0.B.1 Program Operational Semantics
For any given program, we assume that the program semantics is fixed in the static analyzer and defined by a small-step operational semantics [35] specifying the program states , the initial states , the actions , and the transition relation . This is used to generate a trace operational semantics , which is the set of maximal finite or infinite traces starting from an initial state in and such that any two consecutive states are separated by an action as specified by .
The events are chosen to be transitions s\stackrel{{\scriptstyle\hbox{\set@color\hskip 5.0pt\raisebox{-1.07639pt}{\smalla}\hskip 6.60004pt}}}{{\hbox to16.35735pt{\mathord{-}\mkern-7.0mu\cleaders\hbox{}\hfill\mkern-7.0mu\mathord{\rightarrow}}}}s^{\prime}\in\tau. An isomorphic abstraction of the trace operational semantics where states are erased and events are preserved provides the concrete maximal trace semantics where traces are sequences of events as assumed in Section 3.1.
0.B.2 Static Invariance and Eventuality Analyzers
Our responsibility analysis is designed so as to reuse a static invariance analyzer, which is able to attach to each program point an abstract invariant in an abstract domain where the concretization of is false/. Such a static analyzer over-approximates states which are reachable from an initial abstract invariant (e.g. the initial states) and may reach a final abstract invariant (e.g. the final states), by iterating fixpoint forward/backward static analyses in the abstract domain , if necessary with widening extrapolation/narrowing interpolation [14, Section 6]. To be more precise, the forward analysis uses a least fixpoint. An option of the static analyzer is offered to specify whether programs are assumed to terminate (thus excluding infinite traces) or not. Therefore the backward analysis uses a least fixpoint if termination is optionally required and a greatest fixpoint to allow for non-termination.
If available, an existing eventuality static analyzer such as [41, 43] can be used to increase the precision of the responsibility analysis. Such an eventuality static analyzer should be able to prove that there exists a concrete execution which, starting from a reachable state satisfying an abstract invariant, will definitely reach a terminal state satisfying an abstract property (or that this is impossible).
0.B.3 User Specification for the Responsibility Analysis
In general, for a static responsibility analysis where is defined as in Section 0.B.1, the user must specify the lattice of properties of interest , the cognizance , the behavior and traces of interest.
One possibility is to design a static analyzer where and are fixed, while is given by options. Another possibility is to design a parameterized analyzer with parameters specified by the user.
Specifically, here we consider the most frequent case of , which includes a trace property and its complement . The behavior of interest is assumed to be (or alternatively ). Thus, the user needs to specify the sets of traces , , and , as well as the cognizance .
[TABLE]
The user specification needs to be given in the abstract domain . More precisely, , , and are specified by abstract invariants in attached to each program point, which represent sets of maximal concrete traces such that the abstract invariant attached to a program point holds for the program state whenever the trace reaches that program point. If termination is requested only the finite traces are considered.
The cognizance function is specified by abstract relational invariants in expressing relational properties about two executions of a single program on different inputs [3]. Such a relational invariant abstracts the relation between traces, in which the states of and at any program point ℓ satisfy the invariant relation attached to that point.
Example 12
Let us take the following program as an example of the abstract responsibility analysis for the bug at line with the omniscient cognizance.
[TABLE]
In this example, we assume that the abstract domain chosen for the static invariance analyzer and user specifications can express the reduced product of intervals and symbolic (dis)equalities. The abstract user specifications are as follows (the superscript u stands for user specification):
[TABLE]
In the above table, specifies that all valid traces are analyzed, specifies the set of erroneous traces in which is 0 at , and specifies the set of correct traces. In addition, two example cases for the cognizance are given. For the omniscient observer with cognizance , the input at is done knowing the value of , thus only identical traces are related by the invariant; for a non-omniscient observer with cognizance , the input at is independent of the input at , thus traces with identical values of are related in the invariant, regardless of the value of . ∎
0.B.4 Strengthening the User Specification by the Abstract Invariance Semantics
The objective of the invariance static analysis is to abstract the program concrete semantics of Section 0.B.1 and to strengthen the user specifications of Section 0.B.3. Let be the abstract invariance semantics over-approximating computed by the static analyzer of Section 0.B.2. Since the user specifications , , and are relative to the concrete trace semantics , they can be strengthened into , , and by intersection with the abstract invariance semantics .
Example 13
Continuing Example 12, an iterated forward reachability/backward accessibility static invariance analyzer can yield the abstract invariance semantics and strengthened user specifications as follows:
[TABLE]
Notice that instead of computing the abstract semantics and then intersecting with , and , it would, in general, be more precise to do four analyses, each of which is for one of these three cases [14, Section 6].
0.B.5 Construction of an Abstract Floyd-Hoare Automaton
For responsibility analysis, we adopt the more refined abstraction of sets of traces by Floyd-Hoare automata [24, 18]. They are a symbolic connected graph/automaton/control flow graph whose nodes/vertices are pairs of a program point ℓ and an abstract invariant belonging to the abstract domain and whose arcs/edges correspond to abstract events/labelled basic program actions from one program point ℓ to another abstracting a concrete event where ℓ is the control state of and that of (the labelling of actions ensure that any action in the graph corresponds to a unique program action ). An Floyd-Haore automaton represents an over-approximation of all concrete traces, in which states satisfy the abstract invariants when following a path in the graph (the condition avoids abstract dead paths in the automaton, but there could be abstract paths with no corresponding concrete execution). For simplicity, program loops are preserved in the graph (but may have to be unrolled for more precision).
The Floyd-Hoare automaton abstracting , , and (see Fig. 7 for Example 12) is built from the program control flow graph, splitting nodes at labels ℓ whenever but still merging at loop heads to ensure finiteness. The program actions are added on the arcs/edges for events , while abstract invariants are added on the nodes/vertices ℓ of the graph. Any subgraph abstracts a trace property. From now on, abstract trace properties are represented by a set of paths in the Floyd-Hoare automaton. For example, of Example 13 is the maximal path marked , , , in Fig. 7 .
0.B.6 Abstract Observation Analysis
Given , , and , the observation analysis consists in computing an abstraction of for all concrete prefix traces in the concretization of an abstract trace from the entry node to node labelled ℓ in the Floyd-Hoare automaton. We consider the case of the omniscient cognizance (so that ) and the behaviors of interest in are all finite. According to the definition of in (3.3.1), for any which is a set of maximal traces, we need to determine the prefix traces such that . This can be done in the abstract using the Floyd-Hoare automaton and made more precise using the eventuality static analyzer.
The objective is to mark every node by which is , , , or . (respectively ) means that from ℓ there exists a reachable concrete state satisfying from which a concrete execution will definitely reach a terminal state satisfying (respectively ). means that there exists a reachable concrete state satisfying from which there are two concrete executions which will definitely reach a terminal state satisfying for one and for the other. means that this is a possibility but not a certitude.
We start with marking every terminal node in the Floyd-Hoare automaton with either or and marking every other node with . Then we proceed backwards iteratively, starting form terminal nodes, by considering all nodes ℓ of the Floyd-Hoare automaton. The node ℓ is marked (respectively ) if all its successors in the Floyd-Hoare automaton are marked (respectively ) and the eventuality static analyzer can prove that this property holds at ℓ (i.e., as explained above, a terminal node satisfying (respectively ) is reachable in the concrete). If the node ℓ has one successor marked and another one marked , then it is marked if the eventuality static analyzer can prove that this property holds at ℓ (i.e., as explained above, a terminal node satisfying and another terminal node satisfying are both definitely reachable in the concrete). Otherwise, the node ℓ is marked . Since the Floyd-Hoare automaton is finite, this fixpoint computation does terminate.
Example 14
Continuing Example 13, the Floyd-Hoare automaton is given in Figure 7. Each node in the automaton is associated with a program point ℓ, and is decorated by an abstract invariant holding at that point and a boxed abstract observation .
At the initialization all nodes are marked but the double-circled circled nodes marked and marked . Then the iteration marks similarly the nodes and . The nodes and are marked if the eventuality analyzer provides a guarantee that and will definitely be reachable. Should that eventuality analysis fail, the mark would be . ∎
0.B.7 Abstract Responsibility Analysis
For an abstract behavior of interest, the responsibility analysis searches each of the paths that satisfy the abstraction of in the Floyd-Hoare automaton forward, from the initial state. This search looks for the first transition from ℓ to where ℓ is marked by and is marked , such that \overline{\mathcal{P}}^{\prime}\sqsubseteq\overline{\mathcal{B}}\mathrel{\hbox to0.0pt{\raisebox{-2.15277pt}[0.0pt][0.0pt]{\tiny\mskip 7.0mu\scriptscriptstyle\shortmid}\hss}{\sqsubseteq}}\overline{\mathcal{P}}. For the sake of simplicity, it is assumed that is (or alternatively ) in Section 0.B.3.
For every elementary path of the Floyd/Hoare automaton from the origin to a terminal node marked we look for the first transition from ℓ to where ℓ is marked by and is marked , if any. Then the corresponding action is definitely responsible for on all concretizations of that path.
Else, we look on that elementary path of the Floyd/Hoare automaton for the last transition from ℓ to where ℓ is marked by and is marked , if any. Then the corresponding action is potentially responsible for as well as all actions before on that path which concretization have more than one choice (e.g. input commands). (Notice from ℓ to where ℓ is marked by and by is also the first one since all previous before ℓ on that elementary path will be marked .)
Otherwise, all points ℓ on that elementary path are marked and no action is responsible for on that elementary path of the Floyd/Hoare automaton.
All elementary paths of the Floyd/Hoare automaton must be explored since responsibility may occur in different contexts.
Theorem 0.B.1
The abstract responsibility analysis is sound.
Proof
The proof is included in the appendix 0.B.8.
Example 15
Continuing Example 14, the Floyd-Hoare automaton of Figure 7, determines that, for , on the path , , where , we have and so the action b = input_2() is definitely responsible for the bug (since by omniscient cognizance the result of a = input_1() is known). In accordance with (3.4), the future is ignored in the abstraction, thus the path needs not be explored further (in gray in Figure 7).
If, on the contrary, the eventuality analysis is imprecise and (and therefore ) and , the responsibility analysis determine that action b = input_2() is potentially responsible for the bug and so is the preceding action a = input_1() on that path. ∎
It is important to note that the user specification of is within the abstract domain , so there is no risk that two different properties of interest in have the same abstraction in . For example, if and are abstracted to the same property by a coarser abstraction, the Floyd-Hoare automaton merges and as well as and in Fig. 7, making the responsibility analysis impossible.
Another important remark is that the definition of concrete responsibility (3.4) is definite (i.e. a necessary and sufficient condition). On the contrary, the abstract responsibility proposed in this section may be definite but also potential (i.e. a necessary but maybe not sufficient condition). This ensure that events not responsible in the abstract are definitely not responsible in the concrete. Moreover in loops, abstract responsibility does not report at which loop iteration the responsibility of the event is established.
0.B.8 Proofs for Theorem 0.B.1
We start with an example showing that if an action is declared potentially responsible then it might not be while one of the previous action offering a concrete choice might be.
Example 16
Let us analyze which input is responsible for the bug at line of the following program with omniscient cognizance.
[TABLE]
The static analysis is with intervals and [dis]equalities but we assume the invariance analyzer cannot determine that when reaching line . Moreover the eventuality analysis is assumed to be completely ineffective. The Floyd/Hoare automaton (in absence of eventuality analysis) is given in Figure 8. In the concrete, the responsibility definition (3.4) states that action a = input_1(); is responsible for i.e. c=0. The abstract analysis determines that a = input_2(); is potentially responsible for which is not false (the path , , is dead but this is not known in the abstract). However, not adding the previous actions with multi-choices, in this example a = input_1();, would be unsound. ∎
Theorem 0.B.1 The abstract responsibility analysis is sound.
Proof
For simplicity we assume the omniscient congnizance and the traces of interest are all traces of (otherwise below has to be restricted to the concretization of ).
We assume that the Floyd-Hoare automaton is correct, meaning that every execution trace in the operational semantics is in the concretization of a path in the Floyd-Hoare automaton. Since the operational semantics is abstracted to the event traces in in which the events are chosen to be the transitions s\stackrel{{\scriptstyle\hbox{\set@color\hskip 5.0pt\raisebox{-1.07639pt}{\smalla}\hskip 6.60004pt}}}{{\hbox to16.35735pt{\mathord{-}\mkern-7.0mu\cleaders\hbox{}\hfill\mkern-7.0mu\mathord{\rightarrow}}}}s^{\prime}\in\tau and states are erased, we have that and are isomorphic, so the abstraction of by the Floyd-Hoare automaton is also sound. However, we do not exclude that the static analysis of Section 0.B.2 is imprecise so that the Floyd-Hoare automaton may contain spurious paths.
We also assume that the abstract observation analysis of Section 0.B.6 made in the lattice , , , , is sound. So if a node ℓ is marked (respectively ) in the automaton then there exists a concrete execution in through a state at ℓ to a terminal node state satisfying (respectively to ). For a node ℓ marked , we assume that the eventuality analysis has determined that there exists a concrete execution in reaching a state at ℓ that can be continued both to a terminal node state satisfying and to another terminal node state satisfying . This is not guaranteed if the node ℓ is marked .
Consider in (3.4) so that the action of \sigma_{\textup{R}}={{}^{\ell}}\stackrel{{\scriptstyle\hbox{\set@color\hskip 5.0pt\raisebox{-1.07639pt}{\smalla}\hskip 6.60004pt}}}{{\hbox to16.35735pt{\mathord{-}\mkern-7.0mu\cleaders\hbox{}\hfill\mkern-7.0mu\mathord{\rightarrow}}}}{{}^{\ell^{\prime}}} will be declared responsible for in the concrete. We must show that this action will also be declared responsible in the Floyd-Hoare automaton.
By Lemma 4, there exists in heading to . By soundness of the Floyd-Hoare automaton, there are paths to and to in the automaton respectively generating, by concretization, and .
Since the events are transitions s\stackrel{{\scriptstyle\hbox{\set@color\hskip 5.0pt\raisebox{-1.07639pt}{\smalla}\hskip 6.60004pt}}}{{\hbox to16.35735pt{\mathord{-}\mkern-7.0mu\cleaders\hbox{}\hfill\mkern-7.0mu\mathord{\rightarrow}}}}s^{\prime}\in\tau and abstract events are labelled action where ℓ and are the unique labels in the automaton abstracting and respectively, and since and both abstract , we have . There are two cases.
If all nodes on are marked or then, by Section 0.B.7, the action on which is the same as that on is declared definitely responsible, as required.
Otherwise, if some node on is marked then, since the end of the path is marked there is some later action on in the Floyd-Hoare automaton which is marked potentially responsible. But then all actions on that path offering a choice in the concrete may also be responsible so action will be declared potentially responsible.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Abadi, M., Banerjee, A., Heintze, N., Riecke, J.G.: A core calculus of dependency. In: POPL. pp. 147–160. ACM (1999)
- 2[2] Agrawal, H., Horgan, J.R.: Dynamic program slicing. In: PLDI. pp. 246–256. ACM (1990)
- 3[3] Aguirre, A., Barthe, G., Gaboardi, M., Garg, D., Strub, P.: A relational logic for higher-order programs. PACMPL 1 (ICFP), 21:1–21:29 (2017)
- 4[4] Ball, T., Naik, M., Rajamani, S.K.: From symptom to cause: localizing errors in counterexample traces. In: POPL. pp. 97–105. ACM (2003)
- 5[5] Beebee, H., Hitchcock, C., Menzie, P.: The Oxford Handbook of Causation. Oxford University Press (2009)
- 6[6] Beer, I., Ben-David, S., Chockler, H., Orni, A., Trefler, R.J.: Explaining counterexamples using causality. Formal Methods in System Design 40 (1), 20–40 (2012)
- 7[7] Chen, B., Pearl, J., Bareinboim, E.: Incorporating knowledge into structural equation models using auxiliary variables. In: IJCAI. pp. 3577–3583. IJCAI/AAAI Press (2016)
- 8[8] Cheney, J., Ahmed, A., Acar, U.A.: Provenance as dependency analysis. Mathematical Structures in Computer Science 21 (6), 1301–1337 (2011)