Detecting Fault Injection Attacks with Runtime Verification

TL;DR

This paper introduces formal runtime verification monitors designed to detect fault injection attacks, including test inversion and control flow jumps, providing a lightweight and deployable security solution validated on a PIN verification program.

Contribution

It presents formal models of runtime monitors for fault injection detection, with proofs of effectiveness and flexible deployment options, advancing security measures against fault injection attacks.

Findings

Monitors effectively detect simulated fault injection attacks.

Monitors are small, formal, and can be deployed flexibly.

Validation on a PIN verification program confirms effectiveness.

Abstract

Fault injections are increasingly used to attack/test secure applications. In this paper, we define formal models of runtime monitors that can detect fault injections that result in test inversion attacks and arbitrary jumps in the control flow. Runtime verification monitors offer several advantages. The code implementing a monitor is small compared to the entire application code. Monitors have a formal semantics; and we prove that they effectively detect attacks. Each monitor is a module dedicated to detecting an attack and can be deployed as needed to secure the application. A monitor can run separately from the application or it can be ``weaved'' inside the application. Our monitors have been validated by detecting simulated attacks on a program that verifies a user PIN.

Tables

Table 1. Table 1: Cumulative execution time and memory footprint of 100K runs of 𝙿 𝚎𝚡𝚎𝚌 𝟹 subscript 𝙿 subscript 𝚎𝚡𝚎𝚌 3 \mathtt{P_{exec_{3}}} .

	CPU Time (ms)	Memory (KB)
verifyPIN	164	284
Inst. verifyPIN	209 ($\times$ 1.27)	796.6 ($\times$ 2.8)
Inst. verifyPIN & Monitors	228 ($\times$ 1.39)	801 ($\times$ 2.82)

Equations

$$\mathtt{eT}({\mathtt{I}}_1,3,0).\mathtt{eT}({\mathtt{I}}_1,3,0)$$

$$.\mathtt{eT}({\mathtt{I}}_2,BOOL\_FALSE,BOOL\_TRUE)$$

$$\mathtt{i}\mapsto {\mathtt{I}}_1$$

$$\mathtt{i}\mapsto {\mathtt{I}}_2$$

$$\mathtt{begin}({\mathtt{I}}_1).\mathtt{begin}({\mathtt{I}}_1).\mathtt{end}({\mathtt{I}}_1).\mathtt{end}({\mathtt{I}}_1).\mathbf{begin}({\mathbf{I}}_2).\mathbf{begin}({\mathbf{I}}_2).\mathtt{begin}({\mathtt{I}}_3)$$

$$.\mathtt{begin}({\mathtt{I}}_3).\mathtt{end}({\mathtt{I}}_3).\mathtt{end}({\mathtt{I}}_3).\mathtt{begin}({\mathtt{I}}_5).\mathtt{begin}({\mathtt{I}}_5).\mathtt{end}({\mathtt{I}}_5).\mathtt{end}({\mathtt{I}}_5)$$

$$\mathtt{i}\mapsto {\mathtt{I}}_1$$

$$\mathtt{i}\mapsto {\mathtt{I}}_2$$

$$\mathtt{i}\mapsto {\mathtt{I}}_3$$

$$\mathtt{i}\mapsto {\mathtt{I}}_5$$

Full text

K****eywords Runtime Verification, Monitor, Fault Injection, Detection, Quantified Event Automata, Attacker Model

1 Introduction

Fault injections are effective techniques to exploit vulnerabilities in embedded applications and implementations of cryptographic primitives [1, 2, 3, 4, 5, 6]. Fault injections can be thwarted (or detected) using software or hardware countermeasures [7, 8]. Hardware countermeasures are expensive and unpractical for off-the-shelf products. Henceforth, software countermeasures are commonly adopted. Software countermeasures can be categorized into algorithm-level and instruction-level countermeasures.

Most of the existing algorithm-level countermeasures make use of redundancy such as computing a cryptographic operation twice then comparing the outputs [9], or using parity bits [10]. Algorithm-level countermeasures are easy to implement and deploy and usually introduce low (computational and footprint) overhead. However, as they are usually based on redundancy, they can be broken using multiple faults injection or by skipping critical instruction [11, 12].

In the other hand, instruction-level countermeasures require changes to the low level instruction code, for example by applying instruction duplication or triplication [13, 14]. Instruction-level countermeasures are more robust than algorithm-level countermeasures. Indeed, it is believed that instruction-level countermeasures are secure against multiple faults injection under the assumption that skipping two consecutive instructions is too expensive and requires high synchronization capabilities [13, 15]. However, instruction-level countermeasures introduce a large overhead, for instance, instruction duplication doubles the execution time. Moreover, instruction-level countermeasures require changing the instruction set (with dedicated compilers) since e.g., some instructions have to be replaced by a sequence of idempotent or semantically equivalent instructions.

In this paper, we use runtime verification principles [16, 17, 18, 19] and monitors to detect fault injections that result in test inversion or unexpected jumps in the control flow of a program. A monitor can run off-line after the end of an execution provided that the necessary events have been saved in a safe log, or on-line in parallel with the execution. We use Quantified Event Automata (QEAs) [20] to express our monitors. We prove that our monitors for test inversion and jump attacks detect them if and only if such attack occur at runtime – Propositions 1 and 2, respectively. From an implementation point of view, we validate Java implementations of our monitors using attack examples on a program that verifies a user PIN code taken from the FISSC benchmark [21]. Our monitors are lightweight and small in size. The monitors execution time is proportional to the size of the program under surveillance. However, the memory overhead can be bounded as only data related to the “active” basic block need to be kept in memory.

This make them suitable for low-resource devices where monitors can run in a small (hardware)-secured memory (where the entire application may not fit). Moreover, monitors can run separately from the application under surveillance, and thus can run remotely (in a secure environment) provided a secure communication channel.

The rest of the paper is structured as follows. Section 2 overviews Quantified Event Automata. Section 3 introduces preliminaries. Section 4 introduces execution and attacker models. Section 5 introduces our monitors. Section 6 describes an experiment that validates the effectiveness of our monitors. Section 7 discusses related work. Finally, Section 8 concludes and outlines avenues for future work.

2 Quantified Event Automata

We briefly overview Quantified Event Automata [20] (QEAs) which are used to express monitors. QEAs are an expressive formalism to represent parametric specifications to be checked at runtime. An Event Automaton (EA) is a (possibly non-deterministic) finite-state automaton whose alphabet consists of parametric events and whose transitions may be labeled with guards and assignments. The syntax of EA is built from a set of event names $\mathcal{N}$, a set of values $\mathtt{Val}$, and a set of variables $\mathtt{Var}$ (disjoint from $\mathtt{Val}$). The set of symbols is defined as $\mathtt{Sym}=\mathtt{Val}\cup\mathtt{Var}$. An event is a tuple $\langle\mathtt{e,p_{1},\ldots,p_{n}}\rangle$, where $\mathtt{e}\in\mathcal{N}$ is the event name and $\mathtt{p_{1},\ldots,p_{n}}\in\mathtt{Sym}^{\mathtt{n}}$ are the event parameters. We use a functional notation to denote events: $\langle\mathtt{e,p_{1},\ldots,p_{n}}\rangle$ is denoted by $\mathtt{e(p_{1},\ldots,p_{n})}$. Events that are variable-free are called ground events, i.e., an event $\mathtt{e(p_{1},\ldots,p_{n})}$ is ground if $\mathtt{p_{1},\ldots,p_{n}}\in\mathtt{Val}^{\mathtt{n}}$. A trace is defined as a finite sequence of ground events. We denote the empty trace by $\epsilon$.

The semantics of an EA is close to the one of a finite-state automaton with the natural addition of guards and assignments on transitions. A transition can be triggered only if its guard evaluates to $\mathtt{True}$ with the current binding (a map from variables to concrete values), and the assignment updates the current binding.

A QEA is an EA with some (or none) of its variables quantified by $\forall$ or $\exists$. Unquantified variables are left free and they can be manipulated through assignments and updated during the processing of the trace. A QEA accepts a trace if after instantiating the quantified variables with the values derived from the trace, the resulting EAs accept the trace. Each EA consumes only a certain set of events, however a trace can contain other events which are filtered out. The quantification $\forall$ means that a trace has to be accepted by all EAs, while the quantification $\exists$ means that it has to be accepted by at least one EA. For a QEA $\mathtt{M}$ with quantified variables $\mathtt{x_{1},\ldots,x_{n}}$. We use the functional notation $\mathtt{M(x_{1},\ldots,x_{n})}$ to refer to the related EAs depending on the values taken by $\mathtt{x_{1},\ldots,x_{n}}$.

We depict QEAs graphically. The initial state of a QEA has an arrow pointing to it. The shaded states are final states (i.e., accepting states), while white states are failure states (i.e., non-accepting states). Square states are closed-to-failure, i.e., if no transition can be taken then there is an implicit transition to a (sink) failure state. Circular states are closed-to-self (aka skip-states), i.e., if no transition can be taken, then there is an implicit self-looping transition. We use the notation $\frac{\mathit{guard}}{\mathit{assignment}}$ to write guards and assignments on transitions, $:=$ for variable assignment, and $==$ for equality test.

Example 1** (QEA).**

Figure 1 depicts $\mathtt{M_{R}}$, a QEA that checks whether a given trace satisfies requirement $\mathtt{R}$: “for all $\mathtt{i}$, event $\mathtt{e_{1}(i)}$ should precede event $\mathtt{e_{2}(i)}$”. The alphabet of $\mathtt{M_{R}}$ is $\Sigma_{\mathtt{R}}=\{\mathtt{e_{1}(i)},\mathtt{e_{2}(i)}\}$. Consequently, any event in an input trace that is not an instantiation of $\mathtt{e_{1}(i)}$ or $\mathtt{e_{2}(i)}$ is ignored by $\mathtt{M_{R}}$. QEA $\mathtt{M_{R}}$ has two states (1) and (2), which are both accepting states, one quantified variable $\mathtt{i}$, and zero free variables. As its initial state (1) is an accepting state, the empty trace is accepted by $\mathtt{M_{R}}$. State (1) is a square state, hence an event $\mathtt{e_{2}(i)}$ at state (1) leads into an implicit failure state, which is what we want as it would not be preceded by event $\mathtt{e_{1}(i)}$ in this case. An event $\mathtt{e_{1}(i)}$ at state (1) leads into state (2) which is a skipping state, so after an occurrence of event $\mathtt{e_{1}(i)}$ any sequence of events (for the same value of $\mathtt{i}$) is accepted. We note that one can equivalently replace state (2) with an accepting square state with a self-loop labeled by $\Sigma_{\mathtt{R}}$.

Quantification $\forall\mathtt{i}$ means that the property must hold for all values that $\mathtt{i}$ takes in a trace. Each instantiation of $\mathtt{i}$ results in an EA.

To decide whether a trace is accepted by $\mathtt{M_{R}}$ or not, the trace is first sliced based on the values that can match $\mathtt{i}$. Then, each slice is checked against the event automaton instantiated with the appropriate value for $\mathtt{i}$. For instance, the trace $\mathtt{e_{1}(I_{1})}.\mathtt{e_{2}(I_{2})}.\mathtt{e_{2}(I_{1})}.\mathtt{e_{1}(I_{2})}$ is sliced into the following two slices: $\mathtt{i}\mapsto\mathtt{I_{1}}:\mathtt{e_{1}(I_{1})}.\mathtt{e_{2}(I_{1})}$, and $\mathtt{i}\mapsto\mathtt{I_{2}}:\mathtt{e_{2}(I_{2})}.\mathtt{e_{1}(I_{2})}$. Each slice is checked independently. The slice associated with $\mathtt{I_{1}}$ is accepted by $\mathtt{M_{R}(I_{1})}$ as it ends in the accepting state (2), while the slice associated with $\mathtt{I_{2}}$ is rejected by $\mathtt{M_{R}(I_{2})}$ since the event $\mathtt{e_{2}(I_{2})}$ at state (1) leads to an implicit failure state. Therefore, the whole trace is rejected by the QEA because of the universal quantification on $\mathtt{i}$.

3 Preliminaries and Notations

As a running example, we consider function verifyPIN which is depicted in Listing 1111 The code is inspired from the C version in the FISSC benchmark [21]. . Function verifyPIN is the main function for the verification of a user PIN. It handles the counter of user trials (variable g_ptc), which is initialized to 3, i.e., the user is allowed for 3 trials (one trial per execution). The user is authenticated if the value of g_ptc is greater than 0, and function byteArrayCompare returns BOOL_TRUE. Function byteArrayCompare returns BOOL_TRUE if g_userPin and g_cardPin are equal. Note that BOOL_TRUE and BOOL_FALSE have the values 0xAA and 0x55, respectively. This provides a better protection against faults that modifies data-bytes.

The monitors defined in Section 5 are generic. They are independent from programming language and do not require changes to the low-level code because the required instrumentation to produce events can be made at the the source code level. However, to describe attacks at a lower level, we make use of the three-address code (TAC) representation. TAC is an intermediate-code representation which reassembles for instance LLVM-IR. TAC is machine independent, easy to generate from source code, and can be easily converted into assembly code. In TAC, a program is a finite sequence of three-address instructions. In particular, an instruction $\mathtt{\mbox{{ifZ} }z\mbox{ {goto} }L}$ is a conditional branch instruction that directs the execution flow to $\mathtt{L}$ if the value of $\mathtt{z}$ is 0 (i.e., false). An instruction goto L is an unconditional branch instruction that directs the execution flow to $\mathtt{L}$. A label $\mathtt{L}$ can be assigned to any instruction in the TAC. Instruction $\mbox{{Push} }\mathtt{x}$ pushes the value of $\mathtt{x}$ onto the stack. Before making a function call, parameters are individually pushed onto the stack from right to left. While $\mbox{{Pop} }\mathtt{k}$ pops $\mathtt{k}$ bytes from the stack; it is used to pop parameters after a function call.

Example 2**.**

Listing 2 depicts the TAC representation of verifyPIN. The variables _t0, _t1 and _t2 are compiler-generated temporaries.

To specify how the program under verification has to be instrumented, in Section 4.1, we refer to the control flow graph (CFG) of the program. The CFG of a program is a representation of all the paths that might be taken during its execution. A CFG is a rooted directed graph ($\mathtt{V,E}$), where $\mathtt{V}$ is a set of nodes representing basic blocks, and $\mathtt{E}$ is a set of directed edges representing possible control flow paths between basic blocks. A CFG has an entry node and one exit node. The entry node has no incoming edges while the exit node has no outgoing edges. A basic block is a maximal sequence $\mathtt{S_{1}\ldots S_{n}}$ of instructions such that

• •

it can be entered only at the beginning, i.e., none of the instructions $\mathtt{S_{2}\ldots S_{n}}$ has a label (i.e., target of a branch), and

• •

it can be exited only at the end, i.e., none of the instructions $\mathtt{S_{1}\ldots S_{n-1}}$ is a branch instruction or a return.

A CFG may contain loops. A loop is a sequence $\mathtt{B_{1},\ldots,B_{n}}$ of basic blocks dominated by the first basic block: $\mathtt{B_{1}}$, and having exactly one back-edge from the last basic block: $\mathtt{B_{n}}$ into $\mathtt{B_{1}}$. Note that in a CFG, a basic block $\mathtt{B}$ dominates a basic block $\mathtt{B^{\prime}}$ if every path from the entry node to $\mathtt{B^{\prime}}$ goes through $\mathtt{B}$. An edge $\mathtt{(B^{\prime},B)}$ is called a back-edge if $\mathtt{B}$ dominates $\mathtt{B^{\prime}}$.

Example 3**.**

Figure 2 depicts the CFG of verifyPIN.

We define a test, in TAC, as an instruction that involves a logical expression, i.e., an instruction of the form $\mathtt{z:=(x\>\mathtt{oprel}\>y)}$ where $\>\mathtt{oprel}\>$ is a logical operator, followed by the conditional branch instruction $\mathtt{\mbox{{ifZ} }z\mbox{ {goto} }L}$ for some label L. For a test $\mathtt{A}$ we use the notation $\mathtt{cond(A)}$ to refer to the condition (i.e., the logical expression) involved in $\mathtt{A}$.

Example 4** (Test).**

Function verifyPIN contains two tests:

• •

_t0 := (g_ptc > 0)**, ifZ _t0 goto L1

• •

_t2 := (_t1 == BOOL_TRUE)**, ifZ _t2 goto L2.

Let $\mathtt{B}$ be a basic block. We use the notation $\mathtt{B_{L}}$ to refer to the successor of $\mathtt{B}$ whose head is labeled by L if the tail of $\mathtt{B}$ is an unconditional jump instruction goto L. Similarly, we use the notation $\mathtt{B_{T}}$ (resp. $\mathtt{B_{F}}$) to refer to the successor of $\mathtt{B}$ that is executed when $\mathtt{(x\>\mathtt{oprel}\>y)=True}$ (resp. $\mathtt{(x\>\mathtt{oprel}\>y)=False}$) if $\mathtt{B}$ ends with a test $\mathtt{A}$ where $\mathtt{cond(A)}=\mathtt{(x\>\mathtt{oprel}\>y)}$. Note that $\mathtt{B_{F}}$ is the basic block whose head is labeled by L.

Finally, in what follows, variable $\mathtt{i}$ is used to denote the unique identifier of a basic block $\mathtt{B}$, and $\mathtt{i_{\square}}$ is used to denote the unique identifier of a basic block $\mathtt{B}_{\square}$.

4 Modeling

We define our execution model in Section 4.1. Then, in Section 4.2, we define an attacker model for test inversion and jump attacks.

4.1 Modeling Execution

We define a program execution (a program run) by a finite sequence of events, a trace. Such event-based modeling of program executions is appropriate for monitoring actual events of the program. Events mark important steps in an execution. We consider parametric events of the form $\mathtt{e(p_{1},\ldots,p_{n})}$, where $\mathtt{e}$ is the event name, and $\mathtt{p_{1},\ldots,p_{n}}$ is the list of symbolic parameters that take some concrete values at runtime.

As we consider fault injection attacks, then events themselves are under threat (e.g., an attacker may skip an event emission). Skipping an event may result in a monitor reporting a false attack. In order to ensure events emission, we assume that the program under verification is instrumented so that every event is consecutively emitted twice.

We define the following events which have to be emitted, two consecutive times each, during a program execution, where $\mathtt{G}$ = ($\mathtt{V,E}$) is the CFG of the program:

• •

For every basic block $\mathtt{\mathtt{B}\in V}$, event $\mathtt{begin(i)}$ has to be emitted at the beginning of $\mathtt{B}$.

• •

For every basic block $\mathtt{\mathtt{B}\in V}$, event $\mathtt{end(i)}$ has to be emitted:

• –

just before instruction return, if the tail of $\mathtt{B}$ is return.

• –

at the beginning of $\mathtt{B_{L}}$, if the tail of $\mathtt{B}$ is an unconditional jump instruction goto L. Note that, in this case, events $\mathtt{end(i)}$ have to be emitted before event $\mathtt{begin(i_{L})}$.

• –

at the beginning of both $\mathtt{B_{T}}$ and $\mathtt{B_{F}}$, if the tail of $\mathtt{B}$ is a conditional jump instruction $\mathtt{\mbox{{ifZ} }z\mbox{ {goto} }L}$. Note again that, in this case, events $\mathtt{end(i)}$ have to be emitted before events $\mathtt{begin(i_{T})}$ and $\mathtt{begin(i_{F})}$.

• –

at the end of $\mathtt{B}$, otherwise.

• •

For every loop $\mathtt{B_{1},\ldots,B_{n}}$ in graph $\mathtt{G}$, events $\mathtt{reset(i_{1}),\ldots}$, $\mathtt{reset(i_{n})}$ have to be emitted at the end of $\mathtt{B_{n}}$. Event $\mathtt{reset(i)}$ means that the basic block whose identifier is $\mathtt{i}$ may be executed again as it is involved in a loop. Note that, in this case, $\mathtt{reset}$ events have to be emitted before event $\mathtt{end(i_{n})}$ as the latter is used to detect jump attacks.

• •

For every basic block $\mathtt{\mathtt{B}\in V}$ that ends with a test $\mathtt{A}$, events $\mathtt{bT(i,x,y)}$ and $\mathtt{bF(i,x,y)}$ have to be emitted at the beginning of $\mathtt{B_{T}}$ and $\mathtt{B_{F}}$, respectively, where $\mathtt{cond(A)}=\mathtt{(x\>\mathtt{oprel^{i}}\>y)}$. We note that, in this case, the identifier $\mathtt{i}$ also identifies the test $\mathtt{A}$ and the logical operator $\>\mathtt{oprel^{i}}\>$ as a basic block can contain at most one test.

We define a program execution as follows.

Definition 1**.**

(Program Execution).* Let $\mathtt{P}$ be a program. An execution $\mathtt{P_{exec}}$ of $\mathtt{P}$ is a finite sequence of events $\mathtt{e_{1}.\cdots.e_{n}}$, where $\mathtt{n}\in\mathbb{N}$, such that $\mathtt{e_{j}}\in\Sigma_{\mathtt{ALL}}$ = $\{\mathtt{begin(i)}$, $\mathtt{end(i)}$, $\mathtt{reset(i)}$, $\mathtt{bT(i,x,y)}$, $\mathtt{bF(i,x,y)}\}$ for every $\mathtt{j\in\{1,\ldots,n\}}$.*

For an execution $\mathtt{P_{exec}}$, we use the functional notation $\mathtt{P_{exec}(i)}$ to refer to the trace obtained from $\mathtt{P_{exec}}$ by considering only the related events depending on the values taken by $\mathtt{i}$. Indeed, $\mathtt{P_{exec}(i)}$ contains only the event related to the basic block identified by $\mathtt{i}$.

4.2 Modeling Attacker

We focus on test inversion and jump attacks. A test inversion attack is an attack where the result of a test is inverted. Whereas, a jump attack is an attack that directs the control flow of a program execution in a way that results in a path that does not exist inside the CFG of the program.

Test inversion and jump attacks can be performed using physical means [8], such as voltage and clock glitches, and electromagnetic and laser perturbations, to disturb program executions. An attack can also result from transient errors or malicious software.

We consider the multiple fault injection model for test inversion attacks, whereas we consider the single fault injection model for jump attacks. Indeed, the scenario where a jump from a basic block $\mathtt{B}$ into a basic block $\mathtt{B}^{\prime}$ that is directly followed by a jump from $\mathtt{B}^{\prime}$ into $\mathtt{B}$ may not be detected by our monitors. Note that the limitation of our monitors in detecting jump attacks in case of multiple fault injections is restricted to the case where the injections result in multiple jump attacks. Nevertheless, scenarios where there is only one jump attack and (possibly) other attacks, such as test inversion attack and event skip attack (provided that at most one occurrence of an event is skipped), can be detected by our monitors.

Furthermore, we assume that the attacker can skip at most one of the two consecutive occurrences of an event. Otherwise, monitors may not receive all the necessary events to output correct verdicts.

Test inversion attack.

Consider a basic block $\mathtt{B}$ that ends with a test $\mathtt{A}$ = $\mathtt{z:=(x\>\mathtt{oprel}\>y)}$, $\mathtt{\mbox{{ifZ} }z\mbox{ {goto} }L}$. There is a test inversion attack on $\mathtt{A}$ when $\mathtt{B_{T}}$ is executed when $\mathtt{(x\>\mathtt{oprel}\>y)=False}$, or when $\mathtt{B_{F}}$ is executed when $\mathtt{(x\>\mathtt{oprel}\>y)=True}$. In practice, the result of $\mathtt{A}$ can be inverted, for example, by:

• •

skipping the conditional branch instruction, so that $\mathtt{B_{T}}$ is executed regardless whether $\mathtt{(x\>\mathtt{oprel}\>y)}$ evaluates to $\mathtt{True}$ or $\mathtt{False}$),

• •

skipping the instruction that involves the logical expression provided that variable $\mathtt{z}$ already holds the value that results in branch inversion, or

• •

flipping the value of $\mathtt{z}$ after the logical expression being evaluated.

Definition 2**.**

(Test Inversion Attack).* Let $\mathtt{P}$ be a program, and let $\mathtt{P_{exec}}$ = $\mathtt{e_{1},\ldots,e_{n}}$ be an execution of $\mathtt{P}$. We say that there is a test inversion attack on $\mathtt{P_{exec}}$ if it violates $\mathtt{R_{1}}$ or $\mathtt{R_{2}}$ which are defined as follows, where $\mathtt{i}$ identifies $\>\mathtt{oprel^{i}}\>$:*

• •

$\mathtt{R_{1}}$: for every $\mathtt{j}$, if $\mathtt{e_{j}}=\mathtt{eT(i,x,y)}$ then $\mathtt{(x\>\mathtt{oprel^{i}}\>y)=True}$.

• •

$\mathtt{R_{2}}$: for every $\mathtt{j}$, if $\mathtt{e_{j}}=\mathtt{eF(i,x,y)}$ then $\mathtt{(x\>\mathtt{oprel^{i}}\>y)=False}$.

Jump attack.

In our model, a jump attack interrupts an execution of a basic block, starts an execution of a basic block not at its first instruction, or results in an edge that does not exist in the CFG. In practice, a jump attack can be performed, for example, by manipulating the target address of a branch or return. Note that we do not consider intra-basic block jumps (which are equivalent to skipping one or more instruction inside the same basic block).

Let $\mathtt{B}$ be a basic block, and consider only events $\mathtt{begin(i)}$ and $\mathtt{end(i)}$. Then, in the absence of jump attacks, an execution of $\mathtt{B}$ results in one of the following traces depending on whether none, one, or two events are skipped (assuming events duplication):

• •

$\mathtt{tr_{1}=begin(i)}$.$\mathtt{begin(i)}$.$\mathtt{end(i)}$.$\mathtt{end(i)}$,

• •

$\mathtt{tr_{2}=begin(i)}$.$\mathtt{end(i)}$.$\mathtt{end(i)}$,

• •

$\mathtt{tr_{3}=begin(i)}$.$\mathtt{begin(i)}$.$\mathtt{end(i)}$, or

• •

$\mathtt{tr_{4}=begin(i)}$.$\mathtt{end(i)}$.

During a program execution, $\mathtt{B}$ may get executed more than once only if it is involved in a loop. In this case, between every two executions of $\mathtt{B}$ event $\mathtt{reset(i)}$ should be emitted.

Definition 3**.**

(Jump Attack).* Let $\mathtt{P}$ be a program, and let $\mathtt{P_{exec}}$ be an execution of $\mathtt{P}$. Let $\mathtt{P^{J}_{exec}(i)}$ = $\mathtt{e_{1},\ldots,e_{n}}$ denote the trace obtained from $\mathtt{P_{exec}(i)}$ by filtering out all the events that are not in $\Sigma_{\mathtt{J}}=\{\mathtt{begin(i)},\mathtt{end(i)},\mathtt{reset(i)}\}$. We say that there is a jump attack on $\mathtt{P_{exec}}$ if there exists $\mathtt{i}$ such that $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{3}}$, $\mathtt{R_{4}}$ or $\mathtt{R_{5}}$, which are defined as follows:*

• •

$\mathtt{R_{3}}$: for every $\mathtt{j}$, if $\mathtt{e_{j}}=\mathtt{begin(i)}$ then

• –

$\mathtt{e_{j+1}}=\mathtt{end(i)}$, if $\mathtt{e_{j-1}}=\mathtt{begin(i)}$.

• –

$\mathtt{e_{j+1}}=\mathtt{end(i)}$, or $\mathtt{e_{j+1}}=\mathtt{begin(i)}$ and $\mathtt{e_{j+2}}=\mathtt{end(i)}$, if $\mathtt{e_{j-1}}\neq\mathtt{begin(i)}$.

• •

$\mathtt{R_{4}}$: for every $\mathtt{j}$, if $\mathtt{e_{j}}=\mathtt{end(i)}$ then

• –

$\mathtt{e_{j-1}}=\mathtt{begin(i)}$, if $\mathtt{e_{j+1}}=\mathtt{end(i)}$.

• –

$\mathtt{e_{j-1}}=\mathtt{begin(i)}$, or $\mathtt{e_{j-1}}=\mathtt{end(i)}$ and $\mathtt{e_{j-2}}=\mathtt{begin(i)}$, if $\mathtt{e_{j+1}}\neq\mathtt{end(i)}$.

• •

$\mathtt{R_{5}}$: there is no $\mathtt{j}$ such that $\mathtt{e_{j}}=\mathtt{end(i)}$ and $\mathtt{e_{j+1}}=\mathtt{begin(i)}$.

Definition 3 considers the jump attacks that result in executions that cannot be built by concatenating elements from {$\mathtt{tr_{1}}$, $\mathtt{tr_{2}}$, $\mathtt{tr_{3}}$, $\mathtt{tr_{4}}$, $\mathtt{reset(i)}$}. Namely, it considers the following attacks:

• •

Any attack that interrupts an execution of a basic block $\mathtt{B}$. This attack results in one or two consecutive occurrences of event $\mathtt{begin(i)}$ that is not directly followed by event $\mathtt{end(i)}$, which violates requirement $\mathtt{R_{3}}$ of Definition 3.

• •

Any attack that starts the execution of a basic block $\mathtt{B}$ not from its beginning. This attack results in event $\mathtt{end(i)}$ that is not directly preceded by event $\mathtt{begin(i)}$, which violates requirement $\mathtt{R_{4}}$ of Definition 3.

• •

Any attack that performs a backward jump from the end of a basic block $\mathtt{B}_{2}$ into the beginning of a basic block $\mathtt{B}_{1}$ (i.e., the execution already went through $\mathtt{B}_{1}$ before reaching $\mathtt{B}_{2}$) such that there is no edge from $\mathtt{B}_{2}$ to $\mathtt{B}_{1}$ inside the related CFG. This attack results in two executions of $\mathtt{B}_{1}$ that are not separated by, at least, an emission of event $\mathtt{reset(i)}$. Thus, it results in event $\mathtt{end(i)}$ that is directly followed by event $\mathtt{begin(i)}$, which violates requirement $\mathtt{R_{5}}$ of Definition 3. Note that similar forward jumps are not considered by Definition 3 as they do not violate $\mathtt{R_{3}}$, $\mathtt{R_{4}}$ nor $\mathtt{R_{5}}$.

Finally, we note that a trace $\mathtt{P^{J}_{exec}(i)}$ that starts with event $\mathtt{reset(i)}$ or contains more than two consecutive occurrences of event $\mathtt{reset(i)}$ does violate any of the requirements $\mathtt{R_{3}}$, $\mathtt{R_{4}}$ and $\mathtt{R_{5}}$. However, such a trace is produced only if there is a basic block $\mathtt{B}^{\prime}$, with $\mathtt{i^{\prime}\neq i}$, that is executed not from its beginning. Consequently, $\mathtt{P^{J}_{exec}(i^{\prime})}$ violates $\mathtt{R_{4}}$ in this case, and thus the attack that can result in such situation is considered by Definition 3.

Example 5** (Number of Events).**

In order to check verifyPIN for test inversion attacks, it has to be instrumented to produce 8 events (4 $\mathtt{bT(i,x,y)}$ events and 4 $\mathtt{bF(i,x,y)}$ events) since verifyPIN contains two tests and every event has to be emitted twice. Whereas, to check verifyPIN for jump attacks, it has to be instrumented to produce 24 events (10 $\mathtt{begin(i)}$ events and 14 $\mathtt{end(i)}$ events) since verifyPIN has 5 basic blocks, contains two conditional branches, and every event has to be emitted twice.

5 Monitors

We propose monitors that check for the presence/absence of test inversion and jump attacks on a given execution.

We assume that each event is consecutively emitted twice. Note that, in the absence of event skip attack, our monitors can still detect test inversion and jump attacks if each event is emitted only once222A smaller monitor can be used for jump attacks in this case, see Figure 4.. Note also that our monitors can be easily modified to report event skip attack by using a variable to count the number of received events or by tracing the visited states333An additional state has to be added to $\mathtt{M_{TI}}$ in this case, see Figure 3..

5.1 A Monitor for Detecting Test Inversions

Figure 3 depicts monitor $\mathtt{M_{TI}}$, a QEA that detects test inversion attacks on a given execution. The alphabet of $\mathtt{M_{TI}}$ is $\varSigma_{\mathtt{TI}}=\{\mathtt{eT(i,x,y)}$, $\mathtt{eF(i,x,y)}\}$. Monitor $\mathtt{M_{TI}}$ has only one state, which is an accepting square state. It fails when event $\mathtt{eT(i,x,y)}$ is emitted while $\mathtt{(x\>\mathtt{oprel^{i}}\>y)=False}$ (i.e., if the requirement $\mathtt{R_{1}}$ is violated), or when event $\mathtt{eF(i,x,y)}$ is emitted while $\mathtt{(x\>\mathtt{oprel^{i}}\>y)=True}$ (i.e., if the requirement $\mathtt{R_{2}}$ is violated). $\mathtt{M_{TI}}$ accepts multiple occurrences of events $\mathtt{eT(i,x,y)}$ and $\mathtt{eF(i,x,y)}$ as long as the related guards hold. Note that parameter $\mathtt{i}$ is used to identify $\mathtt{oprel^{i}}$ and it allows reporting the test that has been inverted in case of failure.

Proposition 1**.**

Let $\mathtt{P}$ be a program and $\mathtt{P_{exec}}$ an execution of $\mathtt{P}$. Monitor $\mathtt{M_{TI}}$ rejects $\mathtt{P_{exec}}$ iff there is a test inversion attack on $\mathtt{P_{exec}}$.

Example 6** (Test Inversion Attack).**

An attacker can perform a test inversion attack on verifyPIN by skipping the second conditional branch: ifZ _t2 goto L2 (Line 10 of Listing 2). This attack allows the attacker to get authenticated with a wrong PIN. Assuming a wrong PIN, function byteArrayCompare returns a value BOOL_FALSE (which is assigned to _t1). Thus, value 0 is assigned to variable _t2 as a result of the logical instruction _t2:=(_t1 == BOOL_TRUE). At this point, if the conditional branch is skipped, the execution branches to $\mathtt{B}_{3}$ (the success branch) instead of $\mathtt{B}_{4}$ (the failure branch) which was supposed to be executed as the value of _t2 is 0. Provided that verifyPIN is instrumented as described in Section 4.1, the faulted execution after filtering out any event that is not in $\Sigma_{\mathtt{IT}}$ is as follows, where $\mathtt{I_{j}}$ is the identifier of $\mathtt{B_{j}}$, and initially we have g_ptc=3:

[TABLE]

Note that $\mathtt{eT(I_{1},3,0)}$ is emitted at the beginning of $\mathtt{B}_{2}$, the success branch of the first test. It takes the arguments $\mathtt{I_{1}}$, 3, and 0 since the corresponding test is inside $\mathtt{B}_{1}$, and the involved condition is $\mathtt{(g\_ptc>0)}$ where $\mathtt{g\_ptc=3}$. On the other hand, $\mathtt{eT(I_{2},BOOL\_FALSE},\mathtt{BOOL\_TRUE})$ is emitted at the beginning of $\mathtt{B}_{3}$, the success branch of the second test. It takes the arguments $\mathtt{I_{2}}$, BOOL_FALSE, and BOOL_TRUE since the corresponding test is inside $\mathtt{B}_{2}$, and the involved condition is $\mathtt{(\_t1==BOOL\_TRUE)}$ where byteArrayCompare returns BOOL_FALSE into _t1 (as g_userPin is a wrong PIN).

The faulted execution is sliced by $\mathtt{M_{IT}}$, based on the values that $\mathtt{i}$ can take, into the following two slices:

[TABLE]

Slice $\mathtt{i}\mapsto\mathtt{I_{1}}$ satisfies both requirements $\mathtt{R_{1}}$ and $\mathtt{R_{2}}$, and thus it is accepted by $\mathtt{M_{TI}(I_{1})}$. While, slice $\mathtt{i}\mapsto\mathtt{I_{2}}$ does not satisfy the requirement $\mathtt{R_{1}}$ as event $\mathtt{eT(I_{2},BOOL\_FALSE,BOOL\_TRUE)}$ is emitted but $\mathtt{(BOOL\_FALSE==BOOL\_TRUE)=False}$, and thus it is rejected by $\mathtt{M_{IT}(I_{2})}$. Indeed, the occurrence of $\mathtt{eT(I_{2},BOOL\_FALSE,BOOL\_TRUE)}$ leads into an implicit failure state since the related guard is not satisfied. Therefore, since slice $\mathtt{i}\mapsto\mathtt{I_{2}}$ is rejected by $\mathtt{M_{IT}(I_{2})}$, the whole faulted execution is rejected by $\mathtt{M_{IT}}$.

5.2 A Monitor for Detecting Jump Attacks

Figure 4 depicts monitor $\mathtt{M_{J}}$, a QEA that detects jump attacks on a given program execution. The alphabet of $\mathtt{M_{J}(i)}$ is $\varSigma_{\mathtt{J}}=\{\mathtt{begin(i)},\mathtt{end(i)},\mathtt{reset(i)}\}$. Monitor $\mathtt{M_{J}}$ covers every basic block $\mathtt{i}$ inside the CFG of the given program. An instantiation of $\mathtt{i}$ results in the EA $\mathtt{M_{J}(i)}$. Note that $\mathtt{M_{J}}$ assumes a single fault injection model.

Proposition 2**.**

Let $\mathtt{P}$ be a program, and let $\mathtt{P_{exec}}$ be an execution of $\mathtt{P}$. Monitor $\mathtt{M_{J}}$ rejects $\mathtt{P_{exec}}$ iff there is a jump attack on $\mathtt{P_{exec}}$.

Indeed, $\mathtt{M_{J}}$ cannot output a final verdict concerning $\mathtt{R_{1}}$ until the end of the execution as event $\mathtt{end(i)}$ may occur at any point in the future. One way to explicitly catch the end of an execution is to include event exit in $\varSigma_{\mathtt{J}}$, and add a transition from state (5), labeled by exit, to a new accepting square state, say state (6). Then, an occurrence of event exit in state (3) means that event $\mathtt{end(i)}$ will definitely not occur, and thus leads to an implicit failure state. A self-loop on state (4) labeled by exit is also required. Note that an occurrence of event exit at state (1) will also lead to failure.

Example 7** (Jump Attack).**

An attacker can perform a jump attack on verifyPIN by modifying the return address of the function byteArrayCompare, for example, into the address of the first instruction (g_ptc := 3) of the basic block $\mathtt{B}_{3}$ (see Figure 2). This attack interrupts the execution of $\mathtt{B}_{2}$, and allows the attacker to get authenticated with a wrong PIN as it skips the test that follows byteArrayCompare. Consequently, $\mathtt{B}_{3}$ (the success branch) will be executed regardless of the value returned by byteArrayCompare. Provided that verifyPIN is instrumented as described in Section 4.1, the faulted execution after filtering out any event that is not in $\Sigma_{\mathtt{J}}$ is as follows, where $\mathtt{I_{j}}$ is the identifier of $\mathtt{B_{j}}$:

[TABLE]

The faulted execution is sliced by $\mathtt{M_{J}}$, based on the values that $\mathtt{i}$ can take, into the following four slices:

[TABLE]

Slices $\mathtt{i}\mapsto\mathtt{I_{1}}$, $\mathtt{i}\mapsto\mathtt{I_{3}}$, and $\mathtt{i}\mapsto\mathtt{I_{5}}$ satisfy requirements $\mathtt{R_{3}}$, $\mathtt{R_{4}}$ and $\mathtt{R_{5}}$. Thus, they are respectively accepted by $\mathtt{M_{J}(I_{1})}$, $\mathtt{M_{J}(I_{3})}$, and $\mathtt{M_{J}(I_{5})}$. However, slice $\mathtt{i}\mapsto\mathtt{I_{2}}$ does not satisfy the requirement $\mathtt{R_{3}}$ as it contains two consecutive occurrences of event $\mathtt{begin(I_{2})}$ that are not followed by event $\mathtt{end(I_{2})}$. Thus it is rejected by $\mathtt{M_{J}(I_{2})}$. Indeed, the first occurrence of event $\mathtt{begin(I_{2})}$ fires the transition from state (1) into state (2), and the second occurrence of $\mathtt{begin(I_{2})}$ fires the transition from state (2) into state (3), see Figure 4. Thus, $\mathtt{M_{J}(I_{2})}$ ends in state (3), which is a failure state. Therefore, since slice $\mathtt{i}\mapsto\mathtt{I_{2}}$ is rejected by $\mathtt{M_{J}(I_{2})}$, the whole faulted execution is rejected by $\mathtt{M_{J}}$.

Note, given a CFG $\mathtt{(V,E)}$, monitor $\mathtt{M_{J}}$ cannot detect an attack where a forward jump from the end of $\mathtt{B\in V}$ into the beginning of $\mathtt{B^{\prime}\in V}$ with $\mathtt{(B,B^{\prime})\notin E}$ is executed. Indeed, in order to detect such a jump, a global monitor with a structure similar to the CFG is needed where basic blocks are replaced by EAs that resemble $\mathtt{M_{J}(i)}$ with the adjustment of reset transitions in accordance with the loops.

6 Monitor Validation

We validate our monitors by demonstrating their effectiveness in detecting simulated attacks against $\mathtt{P}$ = verifyPIN. Following the initial C implementation, we have implemented verifyPIN and the monitors using Java and AspectJ444www.eclipse.org/aspectj/. We have instrumented verifyPIN at the source code level. More precisely, for every required event we have defined an associated function which is called inside verifyPIN at the positions where the event has to be emitted as specified in Section 4.1. The associated functions are used to define pointcuts in AspectJ. When a function is called, i.e., a pointcut is triggered, the corresponding event is fed to the running monitor. The monitor then makes a transition based on its current state and the received event, and reports a verdict. The code segments executed by the monitor (called advices) are woven within the original source files to generate the final source code that is compiled into an executable.

For example, Listing 3 depicts the Java implementation of $\mathtt{M_{TI}}$. Two states are defined: Ok (accepting state) and Error (failure state). Function updateState (Lines 4-19) takes an event and then, after evaluating the condition $\mathtt{(x\>\mathtt{oprel}\>y)}$, it updates the currentState. If the monitor is in state Ok, the state is updated into Error if the received event is eT and the condition evaluates to false, or the received event is eF and the condition evaluates to true. Once state Error is reached, the monitor cannot exit from it. Function currentVerdict (Lines 21-26) emits verdict CURRENTLY_TRUE if the currentState is Ok, whereas it emits verdict FALSE if the currentState is Error.

In what follows, we illustrate about how we carried out our experiment and the obtained results. The experiment was conducted using Eclipse 4.11 and Java JDK 8u181 on a standard PC (Intel Core i7 2.2 GHz, 16 GB RAM).

6.1 Normal Executions

Providing events duplication, running instrumented verifyPIN in the absence of an attacker results in one of the following 3 executions depending on the values of g_ptc and g_userPin:

• •

$\mathtt{P_{exec_{1}}}$ which contains 10 events: 4 events begin, 4 events end and 2 events eF that result from executing $\mathtt{B}_{1}$ and $\mathtt{B}_{5}$ (see Figure 2). This execution is performed when g_ptc $\leq 0$.

• •

$\mathtt{P_{exec_{2}}}$ which contains 20 events: 8 events begin, 8 events end, 2 events eT and 2 events eF that result from executing $\mathtt{B}_{1}$, $\mathtt{B}_{2}$, $\mathtt{B}_{4}$ and $\mathtt{B}_{5}$. This execution is performed when g_ptc $>0$ and g_userPin $\neq$ g_cardPin.

• •

$\mathtt{P_{exec_{3}}}$ which contains 20 events: 8 events begin, 8 events end and 4 events eT that result from executing $\mathtt{B}_{1}$, $\mathtt{B}_{2}$, $\mathtt{B}_{3}$ and $\mathtt{B}_{5}$. This execution is performed when g_ptc $>0$ and g_userPin = g_cardPin.

Table 1 summarizes the cumulative execution time and the memory footprint of 100K runs of $\mathtt{P_{exec_{3}}}$: (i) without instrumentation, (ii) with instrumentation, and (iii) with instrumentation and the monitors. Note that the memory consumption can be bounded as a monitor processes only one event at a time, keeps track only of the current state, and are parametrized by the current executing block.555We verified empirically that the memory consumption is insensitive to the number of events. However, we did not report the numbers because of lack of space.

The time overhead depends on the size of the application and the number of events. The size of verifyPIN is small; hence the measured overhead represents an extreme and unfavorable situation. A more representative measure of the overhead should be done on larger applications, especially because our approach aims at protecting the critical parts of an application instead of protecting every single instruction. Moreover, we note that our implementation is not optimized yet, and that using AspectJ for instrumentation is not the best choice performance-wise. Instrumentation causes most of the overhead (see Table 1). In the future, using tools such as ASM [22] to directly instrument the bytecode would result in a smaller overhead.

6.2 Test Inversion Attack

Function verifyPIN contains two tests. The first test: _t0 := (g_ptc > 0), ifZ _t0 goto L1 is represented, in Java bytecode, using the instruction ifle L1. The instruction ifle L1 compares g_ptc and 0, which are previously loaded into the stack, and performs a branch into L1 if g_ptc is less than or equal to 0. This test can be inverted by replacing ifle L1 with ifgt L1, which performs a branch if g_ptc is greater than 0.

Similarly, the second one: _t2 := (_t1 == BOOL_TRUE), ifZ _t2 goto L2 is represented using the instruction if_icmpne L2, which compares _t1 and BOOL_TRUE, and performs a branch into L2 if they are not equal. This test can be inverted by replacing if_icmpne with if_icmpeq, which performs a branch if the operands are equal.

The binary opcodes of ifle, ifgt, if_icmpne and if_icmpeq are respectively “1001 1110”, “1001 1101”, “1010 0000” and “1001 1111”. Hence, replacing ifle with ifgt (resp. if_icmpne with if_icmpeq) requires modifying 2 bits (resp. 6 bits).

Depending on the values of g_ptc and g_userPin, a test inversion attack can be used, e.g., to force authentication with a wrong PIN or to prevent the authentication with the correct PIN.

As verifyPIN contains two tests, we consider the two following scenarios:

• •

In case of a wrong user PIN in the first trial (i.e., g_ptc = 3), inverting the second test results in a successful authentication. This is the attack presented in Example 6. Note that monitor $\mathtt{M_{TI}}$ reports an attack after processing the first occurrence of event eT corresponding to the second test as the related guard does not hold in this case. That is after receiving 11 events: 4 events begin, 4 events end and 3 events eT. Whereas, the full execution contains 20 events.

• •

In case of a wrong user PIN in the fourth trial (i.e., g_ptc = 0), inverting both tests results in a successful authentication. Again, $\mathtt{M_{TI}}$ reports an attack after processing the first occurrence of event eT. That is after receiving 5 events: 2 events begin, 2 events end and 1 events eT.

Forcing the “success branch”, in the scenarios above, can be also performed by replacing ifle and/or if_icmpne with nop, which is equivalent to instruction skip, and thus results in the execution of the the “success branch” regardless of the operands’ values. Replacing ifle (resp. if_icmpne) with nop (“0000 0000”, in binary) requires modifying 5 bits (resp. 2 bits). Note that replacing ifle or if_icmpne with nop only works with Java 6 or earlier666 Starting from Java 7, the typing system requires a stack map frame at the beginning of each basic block [23]. Thus, a stack map frame is required by every branching instruction. The stack map frame specifies the verification type of each operand stack entry and of each local variable. Replacing if_icmpne with if_icmpeq does not result in a violation of the related stack map frame, however, replacing it with nop does. Hence, in order to simulate the attack by replacing if_icmpne with nop, the stack map frame also has to be modified. .

Our experiment showed that $\mathtt{M_{TI}}$ can detect both attack scenarios presented above.

6.3 Jump Attack

A jump attack can be simulated, in Java bytecode, by replacing an instruction with goto L for a certain line number L. However, this results in an inconsistent stackmap frame for Java 7 and latest versions. Nevertheless, it is possible to simulate the jump attack presented in Example 7 at the source code level 777 Indeed, it is not possible to simulate this attack by modifying the return address of byteArrayCompare. However, one can simulate the effect of goto using break and continue statements. . We note here that the main purpose is not performing the attack, but to validate that $\mathtt{M_{J}}$ can detect jump attacks. The latter is confirmed by our experiment.

The faulted execution presented in Example 7 contains 16 events: 6 events begin, 6 events end and 2 events eT. However, $\mathtt{M_{J}}$ reports an attack after processing the first occurrence of event $\mathtt{end}$ corresponding to $\mathtt{B}_{3}$. That is, after receiving 9 events: 4 events begin, 3 events end and 2 events eT. Note that the execution of $\mathtt{B}_{2}$ has been interrupted before, but $\mathtt{M_{J}}$ cannot report an attack in this case until the end of the execution as event $\mathtt{end}$ may appear at any time in the future.

7 Related Work

This work introduces formal runtime verification monitors to detect fault attacks. Runtime verification/monitoring was successfully applied to several domains, e.g., for monitoring financial transactions [24], monitoring IT logs [25], monitoring electronic exams [26], monitoring smart homes [27].

In the following, we compare our work to the research endeavors that propose software-based protections against attacks. We distinguish between algorithm-level and instruction-level approaches.

7.1 Algorithm-level Approaches

At the algorithm level, there are approaches that use basic temporal redundancy [28, 1, 11, 8, 29] such as computing a cryptographic operation twice then comparing the outputs. There are also approaches that use parity codes [10] and digest values [30].

Some other works make use of signature mechanisms and techniques to monitor executions, such as state automata and watchdog processor, in order to detect errors or protect the executions control flow. For instance, [31, 32, 33] use watchdogs for error detection at runtime. The underlying principle is to provide a watchdog processor [34] with some information about the process (or processor) to be verified. Then, the watchdog concurrently collects the relevant information at runtime. An error is reported when the comparison test between the collected and provided information fails. The watchdog processor can also be used to detect control flow errors, as illustrated in [32], by verifying that the inserted assertions are executed in the order specified by the CFG. Ersoz et al. [31] propose the watchdog task, a software abstraction of the watchdog processor, to detect execution errors using some assertions. Saxena et al. [33] propose a control-flow checking method using checksums and watchdog. To ensure the control flow of a program, a signature is derived from the instructions based on checksum. However, these approaches focus on protecting solely basic blocks and the control flow, and do not protect branch instructions. Whereas our monitors detect attacks on branch instructions. Moreover, using a watchdog processor for monitoring involves larger communication overhead than using runtime verification.

Nicolescu et al. [35] propose a technique (SIED) to detect transient errors such as bit-flip. This technique has been designed to be combined with an instruction duplication approach. It performs comparison checks and uses signatures to protect the intra-block and inter-block control flow, respectively. Relaying on comparison checks make it subject to fault injections that skip the check itself [36, 37]. Note that experiments have shown that SIED cannot detect all bit-flip faults. Later in [38], the authors propose another error-detection mechanism that provides full coverage against single bit-flip faults. These works only consider single bit-flip as a fault model, whereas our definition of the attacks is independent of the technique used to perform the attack.

Sere et al. [39] propose a set of countermeasures based on basic block signatures and security checks. The framework allows developers to detect mutant applications given a fault model, and thus developing secure applications. The countermeasures can be activated by the developer using an annotation mechanism. This approach requires some modifications in the Java Virtual Machine in order to perform the security checks. Bouffard et al. [40] propose an automaton-based countermeasure against fault injection attacks. For a given program, every state of the corresponding automaton corresponds to a basic block of the CFG, and each transition corresponds to an edge, i.e., allowed control flow. Thus the size of the resulting automaton is proportional to the size of the CFG of the program. Whereas, our monitors are lightweight and small in size.

Fontaine et al. [41] propose a model to protect control flow integrity (CFI). The approach relies on instrumenting the LLVM IR code, and then using an external monitor (state automaton) which enforces CFI at runtime. Lalande et al. [42] propose an approach to detect intra-procedural jump attacks at source code level, and to automatically inject some countermeasures. However, these approaches do not consider test inversion attacks.

Algorithm-level approaches do not require changes to the instruction code. However, they are not effective against compile-time modifications. Moreover, it has been shown that they are not robust against multiple fault injections [11, 12] or skipping the critical parts of the code [36, 37]. Furthermore, most of the existing algorithm-level approaches are not generic.

Our approach is based on the formal model of QEAs, which is one of the most expressive and efficient form of runtime monitor [43]. Our approach is also generic as it can be applied to any application and the monitors can be easily tweaked and used in combination with the monitors for the program requirements. Moreover, monitors may run in a hardware-protected memory as they are lightweight and small in size. This provides more protection against synchronized multiple fault injections on both the monitored program and the monitor. Furthermore, runtime monitoring is modular and compatible with the existing approaches, for instance, monitor $\mathtt{M_{TI}}$ can be used to detect fault attacks on the test that compares the outputs when an operation is computed twice.

Note, to ensure the correct extraction of the necessary information (i.e., events) from a running program, we use emission duplication. This may require the duplication of every related instruction as duplication at the source code level may not be sufficient.

7.2 Instruction-level Approaches

At instruction-level, there are approaches that aim at providing fault-tolerance. These include (i) the approaches that apply the duplication or triplication of instructions [13, 14] in order to provide 100% protection against skip fault injections, and (ii) the approaches that rely on replacing every instruction with a sequence of functionally equivalent instructions such that skipping any of these instructions does not affect the outcome [15, 44]. Such approaches provide more guarantees than algorithm-level approaches. Indeed, it is believed that they are robust against multiple fault injections under the assumption that skipping two consecutive instructions is too expensive and requires high synchronization capabilities [13, 15]. However, they require dedicated compilers for code generation. Moreover, approaches that apply the duplication or replacement of instructions are processor dependent as some instructions have to be replaced by a sequence of idempotent or functionally equivalent instructions. Furthermore, they introduce a large overhead in performance and footprint. For instance, instruction duplication increases the overhead at least twice. Nevertheless, the overhead can be decreased by protecting only the critical parts of the code. In comparison, our monitors are easy to implement and deploy, introduce smaller overhead, and are independent of the processor and the complier. Note that, runtime monitoring does not provide 100% protection against all fault attacks. Our monitors can detect test inversion and jump attacks. Providing more guarantees requires more monitors.

There are also approaches that aim at ensuring CFI. Most existing CFI approaches follow the seminal work by Abadi et al. [45], which makes use of a special set of instructions in order to check the source and destination addresses involved in indirect jumps and indirect function calls. CFI approaches do not aim to provide a 100% fault coverage. Instead they aim at providing protections against jump-oriented attacks [46, 47], and return-oriented attacks [48]. A related technique called control flow locking (CFL) has been introduced by Bletsch et al. [49] in order to provide protection against code-reuse attacks. Instead of inserting checks at control flow transfers, CFL locks the corresponding memory address before a control flow transfer, and then unlocks it after the transfer is executed.

Similar to other instruction-level approaches, these CFI approaches requires change to the instruction code, and usually introduce large overhead. Note that detecting jump attacks by our monitors is some sort of reporting CFI violations. Note also that CFI does not deal with test inversion attacks as taking any of the branches after a conditional branch does not violate CFI.

8 Conclusions and Perspectives

We formally define test inversion and jump attacks. Then, we propose monitors expressed as Quantified Event Automata in order to detect these attacks. Our monitors are lightweight and small in size, and they support the duplication of events emission which provides protection against event skip attacks. Finally, we demonstrate the validity of our monitors using attack examples on verifyPIN.

In the future, we will define more monitors to detect additional attacks following the principles exposed in this paper. For example, a monitor that can detect attacks on function calls. We also plan (i) to verify applications larger than verifyPIN, combined with detailed feasibility and performance analysis, (ii) to use a Java bytecode editing tool, such as ASM [22] or JNIF [50], to simulate faults, and (iii) to deploy the monitors on hardware architectures such as smart cards, Raspberry Pi, and microcontrollers based on Arm Cortex-M processor. Furthermore, we consider building a tool for automatic generation of monitors from QEAs, and developing a runtime enforcement [51, 52, 53] framework where some corrective actions and countermeasures are automatically executed and taken respecticely once an attack is detected.

Appendix A Proofs

See 1

Proof.

Assume that $\mathtt{M_{TI}}$ rejects $\mathtt{P_{exec}}$. We have to show that there is a test inversion attack on $\mathtt{P_{exec}}$, i.e., $\mathtt{P_{exec}}$ violates $\mathtt{R_{1}}$ or $\mathtt{R_{2}}$. As $\mathtt{M_{TI}}$ rejects $\mathtt{P_{exec}}$ then there exists $\mathtt{i}$ such that $\mathtt{M_{TI}(i)}$ fails (i.e., ends in a failure state). Thus, $\mathtt{M_{TI}(i)}$ fires a transition into an implicit failure state since $\mathtt{M_{TI}(i)}$ has only one (explicit) state which is an accepting state. This means that $\mathtt{P_{exec}}$ contains, for some $\mathtt{x}$ and $\mathtt{y}$, event $\mathtt{eT(i,x,y)}$ such that $\mathtt{(x\>\mathtt{oprel^{i}}\>y)=False}$ which violates $\mathtt{R_{1}}$, or event $\mathtt{eF(i,x,y)}$ such that $\mathtt{(x\>\mathtt{oprel^{i}}\>y)=True}$ which violates $\mathtt{R_{2}}$. So, $\mathtt{P_{exec}}$ violates $\mathtt{R_{1}}$ or $\mathtt{R_{2}}$, and thus there is a test inversion attack on $\mathtt{P_{exec}}$. Hence, we can conclude for the first direction.

To prove the second direction, we assume that there is a test inversion attack on $\mathtt{P_{exec}}$, and we show that $\mathtt{M_{TI}}$ rejects $\mathtt{P_{exec}}$. If there is a test inversion attack on $\mathtt{P_{exec}}$, then $\mathtt{P_{exec}}$ violates $\mathtt{R_{1}}$ or $\mathtt{R_{2}}$. If $\mathtt{P_{exec}}$ violates $\mathtt{R_{1}}$ then it contains an event $\mathtt{eT(i,x,y)}$ such that $\mathtt{(x\>\mathtt{oprel^{i}}\>y)=False}$, which fires a transition into an implicit failure state as the guard related to event $\mathtt{eT(i,x,y)}$ is not satisfied. Thus, $\mathtt{M_{TI}}$ fails and rejects $\mathtt{P_{exec}}$. If $\mathtt{P_{exec}}$ violates $\mathtt{R_{2}}$ then it contains an event $\mathtt{eF(i,x,y)}$ such that $\mathtt{(x\>\mathtt{oprel^{i}}\>y)=True}$, which fires a transition into an implicit failure state as the guard related to event $\mathtt{eF(i,x,y)}$ is not satisfied. Thus, $\mathtt{M_{TI}}$ fails and rejects $\mathtt{P_{exec}}$. Hence, we conclude for the second direction and the proof is done. ∎

See 2

Proof.

Assume that $\mathtt{M_{J}}$ rejects $\mathtt{P_{exec}}$. We have to show that there is a jump attack on $\mathtt{P_{exec}}$, i.e., there exists $\mathtt{i}$ such that $\mathtt{P^{J}_{exec}(i)}$ = $\mathtt{e_{1},\ldots,e_{n}}$ violates $\mathtt{R_{3}}$, $\mathtt{R_{4}}$ or $\mathtt{R_{5}}$. As $\mathtt{M_{J}}$ rejects $\mathtt{P_{exec}}$ then there exists $\mathtt{i}$ such that EA $\mathtt{M_{J}(i)}$ fails while consuming $\mathtt{P^{J}_{exec}(i)}$. We split according to the cases in which $\mathtt{M_{J}(i)}$ fails.

• •

$\mathtt{M_{J}(i)}$ has encountered event $\mathtt{end(i)}$ in state (1). This means that there exists $\mathtt{j}$ such that $\mathtt{e_{j}=end(i)}$, and that $\mathtt{e_{j-1}}$ is either $\epsilon$ or $\mathtt{reset(i)}$ since state (1) is the initial state and it can only be reached by $\mathtt{reset(i)}$. Thus, $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{4}}$.

• •

$\mathtt{M_{J}(i)}$ ends in state (2). Then, in this case, there exists $\mathtt{j}$ such that $\mathtt{e_{j}=begin(i)}$ since state (2) can only be reached from state (1) through event $\mathtt{begin(i)}$. Moreover, we can deduce that $\mathtt{e_{j+1}}$ is neither $\mathtt{begin(i)}$ nor $\mathtt{end(i)}$. Thus, $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{3}}$.

• •

$\mathtt{M_{J}(i)}$ has encountered event $\mathtt{e_{j}=reset(i)}$ in state (2). State (2) can only be reached from state (1) through event $\mathtt{begin(i)}$. Then, $\mathtt{e_{j-1}=begin(i)}$. So, there exists $\mathtt{e_{j-1}=begin(i)}$ such that $\mathtt{e_{j}}$ is neither $\mathtt{end(i)}$ nor $\mathtt{begin(i)}$ which violates $\mathtt{R_{3}}$.

• •

$\mathtt{M_{J}(i)}$ ends in state (3). Then there exists $\mathtt{j}$ such that $\mathtt{e_{j-1}=}$ $\mathtt{e_{j}=begin(i)}$ because state (2) can only be reached from state (1) and state (3) can only be reached from state (2), both through event $\mathtt{begin(i)}$. Moreover, $\mathtt{e_{j+1}\neq end(i)}$ as there is a transition from state (3) into state (4) labeled by $\mathtt{end(i)}$, but $\mathtt{M_{J}(i)}$ ends in state (3). Thus, $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{3}}$.

• •

$\mathtt{M_{J}(i)}$ has encountered event $\mathtt{reset(i)}$ in state (3). This case is similar to the previous case and violates $\mathtt{R_{3}}$.

• •

$\mathtt{M_{J}(i)}$ has encountered event $\mathtt{begin(i)}$ in state (3). Then there exists $\mathtt{j}$ such that $\mathtt{e_{j-1}=e_{j}=e_{j+1}}$ = $\mathtt{begin(i)}$, which violates $\mathtt{R_{3}}$.

• •

$\mathtt{M_{J}(i)}$ has encountered event $\mathtt{begin(i)}$ in state (4). This means that there exists $\mathtt{j}$ such that $\mathtt{e_{j}=begin(i)}$, and $\mathtt{e_{j-1}=end(i)}$ since state (4) can only be reached by $\mathtt{end(i)}$ from state (2) or state (3), which can be reached only be reached by $\mathtt{begin(i)}$. Thus, $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{5}}$.

• •

$\mathtt{M_{J}(i)}$ has encountered event $\mathtt{end(i)}$ in state (5). Then there exists $\mathtt{j}$ such that $\mathtt{e_{j-1}=e_{j}}$ = $\mathtt{e_{j+1}=end(i)}$ because state (5) can only be reached from state (4) by $\mathtt{end(i)}$, and state (4) can only be reached by $\mathtt{end(i)}$. Thus, we have that $\mathtt{e_{j}=e_{j+1}=}$ $\mathtt{end(i)}$ and $\mathtt{e_{j-1}\neq begin(i)}$, which violates $\mathtt{R_{4}}$.

• •

$\mathtt{M_{J}(i)}$ has encountered event $\mathtt{begin(i)}$ in state (5). This means that there exists $\mathtt{j}$ such that $\mathtt{e_{j}=begin(i)}$ and $\mathtt{e_{j-1}=end(i)}$ since state (5) can only be reached by $\mathtt{end(i)}$. Thus, $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{5}}$.

Hence, there exists $\mathtt{i}$ such that $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{3}}$, $\mathtt{R_{4}}$ or $\mathtt{R_{5}}$, and we can conclude about the first direction.

To prove the second direction, we assume that there exist $\mathtt{i}$ such that $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{3}}$, $\mathtt{R_{4}}$ or $\mathtt{R_{5}}$ and we show that $\mathtt{M_{J}}$ rejects $\mathtt{P_{exec}}$. We split cases according which requirement is violated.

• •

If $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{3}}$, then it contains an event $\mathtt{e_{j}=begin(i)}$ for some integer $\mathtt{j}$ such that:

• –

$\mathtt{e_{j-1}}=\mathtt{begin(i)}$ and $\mathtt{e_{j+1}}\neq\mathtt{end(i)}$. In this case if, before receiving $\mathtt{e_{j-1}}$, $\mathtt{M_{J}(i)}$ was:

in state (1). Then $\mathtt{e_{j-1}}=\mathtt{begin(i)}$ leads into state (2) and $\mathtt{e_{j}}=\mathtt{begin(i)}$ leads into state (3). As $\mathtt{e_{j+1}}\neq\mathtt{end(i)}$ and from state (3) there is only one explicit transition labeled by $\mathtt{end(i)}$, then $\mathtt{M_{J}(i)}$ ends in state (3) (i.e., $\mathtt{e_{j+1}}=\epsilon$) or a transition into an implicit failure state is fired.

• *

in state (2). Then $\mathtt{e_{j-1}}=\mathtt{begin(i)}$ leads into state (3), and $\mathtt{e_{j}}=\mathtt{begin(i)}$ leads into an implicit failure state.

• *

in state (3), (4) or (5). Then $\mathtt{M_{J}(i)}$ fails since none of the states (3), (4) and (5) has an explicit outgoing transition labeled by $\mathtt{begin(i)}$.

• –

or $\mathtt{e_{j-1}}\neq\mathtt{begin(i)}$ and $\mathtt{e_{j+1}}\neq\mathtt{end(i)}$, and “$\mathtt{e_{j+1}}\neq\mathtt{begin(i)}$ or $\mathtt{e_{j+2}}\neq\mathtt{end(i)}$”. This is equivalent to $\mathtt{e_{j-1}}\neq\mathtt{begin(i)}$ and, $\mathtt{e_{j+1}}=\epsilon$ (i.e., $\mathtt{j=n}$) or $\mathtt{e_{j+1}}=\mathtt{reset(i)}$ or “$\mathtt{e_{j+1}}=\mathtt{begin(i)}$ and $\mathtt{e_{j+2}}\neq\mathtt{end(i)}$” since $\Sigma_{\mathtt{J}}=\{\mathtt{begin(i)},\mathtt{end(i)},\mathtt{reset(i)}\}$. In this case if, before receiving $\mathtt{e_{j-1}}$, $\mathtt{M_{J}(i)}$ was:

in state (1). If $\mathtt{e_{j-1}}$ is $\mathtt{end(i)}$, a transition into an implicit failure state is fired. Otherwise, we have that $\mathtt{e_{j-1}}$ is $\mathtt{reset(i)}$ or $\epsilon$ (i.e., $\mathtt{j=1}$), and thus $\mathtt{M_{J}(i)}$ stays in state (1). Then, $\mathtt{e_{j}=begin(i)}$ leads into state (2). Then in state (2), if $\mathtt{e_{j+1}}$ is $\mathtt{reset(i)}$ a transition into an implicit failure state is fired; if $\mathtt{e_{j+1}}$ is $\epsilon$ then $\mathtt{M_{J}(i)}$ ends in failure state (2); if $\mathtt{e_{j+1}}$ is $\mathtt{begin(i)}$ the transition from state (2) into state (3) is fired and, as $\mathtt{e_{j+2}}\neq\mathtt{end(i)}$ in this case, then a transition into an implicit failure state is fired or $\mathtt{M_{J}(i)}$ ends in failure state (3).

• *

in state (2). If $\mathtt{e_{j-1}}$ is $\mathtt{reset(i)}$, a transition into an implicit failure state is fired. Otherwise, we have that $\mathtt{e_{j-1}}=\mathtt{end(i)}$ which leads from state (2) into state (4). Then in state (4), as $\mathtt{e_{j}}=\mathtt{begin(i)}$, a transition into an implicit failure state is fired.

• *

in state (3). Similar to the case of state (2).

• *

in state (4). If $\mathtt{e_{j-1}}$ is $\mathtt{end(i)}$, the transition into state (5) is fired. Then in state (5), as $\mathtt{e_{j}}=\mathtt{begin(i)}$, a transition into an implicit failure state is fired. Otherwise, we have that $\mathtt{e_{j-1}}=\mathtt{reset(i)}$, and thus the transition from state (4) into state (1) is fired. Then in state (1), as $\mathtt{e_{j}}=\mathtt{begin(i)}$, the transition into state (2) is fired. Then in state (2), if $\mathtt{e_{j+1}}$ is $\mathtt{reset(i)}$ a transition into an implicit failure state is fired; if $\mathtt{e_{j+1}}$ is $\epsilon$ then $\mathtt{M_{J}(i)}$ ends in state (2) which is a failure state; if $\mathtt{e_{j+1}}$ is $\mathtt{begin(i)}$ the transition from state (2) into state (3) is fired and, as $\mathtt{e_{j+2}}\neq\mathtt{end(i)}$ in this case, then a transition into an implicit failure state is fired or $\mathtt{M_{J}(i)}$ ends in state (3), which is a failure state.

• *

in state (5). If $\mathtt{e_{j-1}}$ is $\mathtt{reset(i)}$ the reasoning is similar to the case of state (4) when $\mathtt{e_{j-1}}=\mathtt{reset(i)}$. Otherwise, we have that $\mathtt{e_{j-1}}=\mathtt{end(i)}$ which leads into an implicit failure state.

Hence, If $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{3}}$ then $\mathtt{M_{J}(i)}$ fails, and thus $\mathtt{M_{J}}$ rejects $\mathtt{P_{exec}}$.

• •

If $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{4}}$, then it contains an event $\mathtt{e_{j}=end(i)}$ for some integer $\mathtt{j}$ such that

• –

$\mathtt{e_{j-1}}\neq\mathtt{begin(i)}$ and $\mathtt{e_{j+1}}=\mathtt{end(i)}$. In this case if, before receiving $\mathtt{e_{j-1}}$, $\mathtt{M_{J}(i)}$ was:

in state (1). If $\mathtt{e_{j-1}}$ is $\mathtt{reset(i)}$ the self-loop transition over state (1) is fired. Then, as $\mathtt{e_{j}=end(i)}$, a transition into an implicit failure state is fired. Otherwise, if $\mathtt{e_{j-1}}$ is $\mathtt{end(i)}$, then a transition into an implicit failure state is fired. Otherwise, we have that $\mathtt{e_{j-1}}=\epsilon$ (i.e., $\mathtt{j=1}$), and thus $\mathtt{e_{j}=end(i)}$ leads from state (1) into an implicit failure state.

• *

in state (2). If $\mathtt{e_{j-1}}$ is $\mathtt{reset(i)}$ a transition into an implicit failure state is fired. Otherwise, we have that $\mathtt{e_{j-1}}=\mathtt{end(i)}$, and thus the transition from state (2) into state (4) is fired. Then in state (4), event $\mathtt{e_{j}}=\mathtt{end(i)}$ leads into state (5). Then in state (5), event $\mathtt{e_{j+1}}=\mathtt{end(i)}$ leads into an implicit failure state.

• *

in state (3). Similar to the case of state (2).

• *

in state (4). If $\mathtt{e_{j-1}}=\mathtt{reset(i)}$ the transition to state (1) is fired. Then in state (1), $\mathtt{e_{j}=end(i)}$leads into an implicit failure state. Otherwise, we have that $\mathtt{e_{j-1}}=\mathtt{end(i)}$ which fires the transition from state (4) into state (5). Then in state (5), $\mathtt{e_{j}}=\mathtt{end(i)}$ leads into an implicit failure state.

• *

in state (5). If $\mathtt{e_{j-1}}=\mathtt{reset(i)}$ the transition to state (1) is fired. Then $\mathtt{e_{j}=end(i)}$ leads into an implicit failure state. Otherwise, we have $\mathtt{e_{j-1}}=\mathtt{end(i)}$ which fires a transition into an implicit failure state.

• –

or $\mathtt{e_{j+1}}\neq\mathtt{end(i)}$ and $\mathtt{e_{j-1}}\neq\mathtt{begin(i)}$, and “$\mathtt{e_{j-1}}\neq\mathtt{end(i)}$ or $\mathtt{e_{j-2}}\neq\mathtt{begin(i)}$”. This is equivalent to $\mathtt{e_{j+1}}\neq\mathtt{end(i)}$, and $\mathtt{e_{j-1}}=\mathtt{reset(i)}$ or “$\mathtt{e_{j-1}}=\mathtt{end(i)}$ and $\mathtt{e_{j-2}}\neq\mathtt{begin(i)}$” since $\Sigma_{\mathtt{J}}=\{\mathtt{begin(i)},\mathtt{end(i)},\mathtt{reset(i)}\}$. In this case if, before receiving $\mathtt{e_{j-1}}$, $\mathtt{M_{J}(i)}$ was:

in state (1). If $\mathtt{e_{j-1}}$ is $\mathtt{end(i)}$, a transition into an implicit failure state is fired. Otherwise, we have that $\mathtt{e_{j-1}}$ is $\mathtt{reset(i)}$ or $\epsilon$ (i.e., $\mathtt{j=1}$), and thus $\mathtt{M_{J}(i)}$ stays in state (1). Then, as $\mathtt{e_{j}=end(i)}$, a transition into an implicit failure state is fired.

• *

in state (2). In this case $\mathtt{e_{j-1}}$ must be equal to $\mathtt{reset(i)}$ since if $\mathtt{e_{j-1}}=\mathtt{end(i)}$, then $\mathtt{e_{j-2}}\neq\mathtt{begin(i)}$ and thus state (2) cannot be reached. Indeed, state (2) can only be reached from state (1) by $\mathtt{begin(i)}$. In state (2), $\mathtt{e_{j-1}}=\mathtt{reset(i)}$ leads into an implicit failure state.

• *

in state (3). Similar to the case of state (2).

• *

in state (4). If $\mathtt{e_{j-1}}$ is $\mathtt{reset(i)}$ the transition into state (1) is fired. Then in state (1), as $\mathtt{e_{j}}=\mathtt{end(i)}$, a transition into an implicit failure state is fired. Otherwise, we have that $\mathtt{e_{j-1}}=\mathtt{end(i)}$ which leads into state (5). Then in state (5), $\mathtt{e_{j}}=\mathtt{end(i)}$ leads into an implicit failure state.

• *

in state (5). If $\mathtt{e_{j-1}}=\mathtt{reset(i)}$ the transition into state (1) is fired. Then, as $\mathtt{e_{j}}=\mathtt{end(i)}$, a transition into an implicit failure state is fired. Otherwise, we have that $\mathtt{e_{j-1}}=\mathtt{end(i)}$ which leads into an implicit failure state.

Hence, If $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{4}}$ then $\mathtt{M_{J}(i)}$ fails, and thus $\mathtt{M_{J}}$ rejects $\mathtt{P_{exec}}$.

• •

If $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{5}}$, then it contains an event $\mathtt{e_{j}=end(i)}$ for some integer $\mathtt{j}$ such that $\mathtt{e_{j+1}=begin(i)}$. In this case if, before receiving $\mathtt{e_{j}}$, $\mathtt{M_{J}(i)}$ was:

• –

in state (1). Then event $\mathtt{e_{j}=end(i)}$ fires a transition into an implicit failure state.

• –

in state (2). Then event $\mathtt{e_{j}=end(i)}$ fires the transition into state (4). In state (4), $\mathtt{e_{j+1}}=\mathtt{begin(i)}$ fires a transition into an implicit failure state.

• –

in state (3). Similar to the case of state (2).

• –

in state (4). Then $\mathtt{e_{j}=end(i)}$ fires the transition into state (5). In state (5), $\mathtt{e_{j+1}}=\mathtt{begin(i)}$ fires a transition into an implicit failure state.

• –

in state (5). Then $\mathtt{e_{j}=end(i)}$ fires a transition into an implicit failure state.

Hence, if $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{5}}$ then $\mathtt{M_{J}(i)}$ fails, and thus $\mathtt{M_{J}}$ rejects $\mathtt{P_{exec}}$.

Therefore, if there exists $\mathtt{i}$ such that $\mathtt{P^{J}_{exec}(i)}$ violates $\mathtt{R_{3}}$, $\mathtt{R_{4}}$ or $\mathtt{R_{5}}$ then $\mathtt{M_{J}}$ rejects $\mathtt{P_{exec}}$. Thus, we can conclude for the second direction and the proof is done. ∎