An Operational Guide to Monitorability
Luca Aceto, Antonis Achilleos, Adrian Francalanza, Anna, Ing\'olfsd\'ottir, Karoliina Lehtinen

TL;DR
This paper introduces a hierarchical framework for monitorability in runtime verification, explicitly linking properties to monitor guarantees, and unifies existing definitions to improve understanding and design of verification tools.
Contribution
It proposes a monitorability hierarchy with operational and syntactic characterizations, unifying various existing definitions into a rigorous, explicit framework.
Findings
Mapped existing monitorability definitions into the hierarchy
Provided operational and syntactic characterizations for hierarchy levels
Established a unified framework for runtime verification monitorability
Abstract
Monitorability delineates what properties can be verified at runtime. Although many monitorability definitions exist, few are defined explicitly in terms of the guarantees provided by monitors, i.e., the computational entities carrying out the verification. We view monitorability as a spectrum: the fewer monitor guarantees that are required, the more properties become monitorable. We present a monitorability hierarchy and provide operational and syntactic characterisations for its levels. Existing monitorability definitions are mapped into our hierarchy, providing a unified framework that makes the operational assumptions and guarantees of each definition explicit. This provides a rigorous foundation that can inform design choices and correctness claims for runtime verification tools.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSecurity and Verification in Computing · Formal Methods in Verification · Advanced Software Engineering Methodologies
11institutetext: Gran Sasso Science Institute, L’Aquila, Italy 22institutetext: Reykjavik University, Reykjavik, Iceland 33institutetext: University of Malta, Msida, Malta 44institutetext: University of Liverpool, Liverpool, UK
An Operational Guide to Monitorability
L. Aceto 1122
A. Achilleos 22
A. Francalanza 33
A. Ingólfsdóttir 22
K. Lehtinen 44
Abstract
Monitorability delineates what properties can be verified at runtime. Although many monitorability definitions exist, few are defined explicitly in terms of the guarantees provided by monitors, i.e., the computational entities carrying out the verification. We view monitorability as a spectrum: the fewer monitor guarantees that are required, the more properties become monitorable. We present a monitorability hierarchy and provide operational and syntactic characterisations for its levels. Existing monitorability definitions are mapped into our hierarchy, providing a unified framework that makes the operational assumptions and guarantees of each definition explicit. This provides a rigorous foundation that can inform design choices and correctness claims for runtime verification tools.
1 Introduction
Any sufficiently expressive specification language contains properties that cannot be monitored at runtime [33, 35, 23, 21, 27, 2, 5]. For instance, the satisfaction of a safety property (“bad things never happen”) cannot, in general, be determined by observing the (finite) behaviour of a program up to the current execution point; its violation, however, can. Monitorability [13] concerns itself with the delineation between properties that are monitorable and those that are not. Monitorability is paramount for a slew of Runtime Verification (RV) tools, such as those described in [19, 22, 36, 10, 34] (to name but a few), that synthesise monitors from specifications expressed in a variety of logics. These monitors are then executed with the system under scrutiny to produce verdicts concerning the satisfaction or violation of the specifications from which they were synthesised.
Monitorability is crucial for a principled approach because it disciplines the construction of RV tools. It defines, either explicitly or implicitly, a notion of monitor correctness, which guides the automated synthesis. It also delimits the monitorable fragment of the specification logic on which the synthesis is defined: monitors need not be synthesised for non-monitorable specifications. In some settings, a syntactic characterisation of monitorable properties can be identified [27, 1, 5], and used as a core calculus for studying optimisations of the synthesis algorithm. More broadly, monitorability boundaries may guide the design of hybrid verification strategies, which combine RV with other verification techniques (see the work in [2] for an example of this approach).
In spite of its importance, there is no generally accepted notion of monitorability to date. The literature contains a number of definitions, such as the ones proposed in [24, 35, 15, 27, 5]. These differ in aspects such as the adopted specification formalism, e.g., LTL, Street automata, recHML etc., the operational model, e.g., testers, automata, process calculi etc., and the semantic domain, e.g., infinite traces, finite and infinite (finfinite) traces or labelled transition systems. Even after these differences are normalised, many of these definitions are not in agreement: there are properties that are monitorable according to some definitions but not monitorable according to others. More alarmingly, as we will show, frequently cited definitions of monitorability contain serious errors.
This discrepancy between definitions raises the question of which one to adopt to inform one’s implementation of an RV tool, and what effect this choice has on the behaviour of the resulting tool. A difficulty in informing this choice is that few of those definitions make explicit the relationship between the operational model, i.e., the behaviour of a monitor, and the monitored properties. In other words, it is not clear what the guarantees provided by the various monitors mentioned in the literature are, and how they differ from each other.
Example 1.
Consider the runtime verification of a system exhibiting (only) three events over finfinite traces: failure (f), success (s) and recovery (r). One property we may require is that “failure never occurs and eventually success is reached”, otherwise expressed in LTL fashion as . According to the definition of monitorability attributed to Pnueli and Zaks [35] (discussed in Sec. 7), this property is monitorable. However, it is not monitorable according to others, including Schneider [38], Viswanathan and Kim [39], and Aceto et al. [5], whose definition of monitorability coincides with some subset of safety properties.
Contributions.
To our mind, this state of the art is unsatisfactory for tool construction. An RV tool broadly relies on the following ingredients: the input of the tool in terms of the formalism used to describe the specification properties; the executable description of monitors that are the tool’s output and the mapping between the inputs and outputs, i.e., the synthesis function of monitors from specifications. Any account on monitorability should, in our view, shed light on those three aspects, particularly on what it means for the synthesis function and the monitors it produces to be correct. In addition, if this account is flexible enough to incorporate a variety of relationships between specification properties and the expected behaviour of monitors, it can then be used by the tool implementors as a principled foundation to guide their design decisions.
For these reasons, we take the view that monitorability comes on a spectrum. There is a trade-off between the guarantees provided by monitors and the properties that can be monitored with those guarantees. We argue that considering different requirements gives rise to a hierarchy of monitorability—depicted in Fig. 1 (middle)—which classifies properties according to what types of guarantees RV can give for them. At one extreme, anything can be monitored if the only requirement is for monitors to be sound i.e., they should not contradict the monitored specification. However, monitors that are just sound give no guarantees of ever giving a verdict. More usefully, informatively monitorable properties enjoy monitors that reach a verdict for some finite execution; arguably, this is the minimum requirement for making monitoring potentially worthwhile. More stringent requirements can demand this capability to be invariant over monitor executions, i.e., a monitor never reaches a state where it cannot provide a verdict; then we speak of persistently informative monitors. Adding completeness requirements of different strengths, such as the requirement that a monitor should be able to identify all failures and/or satisfactions, yields stronger definitions of monitorability: partial, satisfaction or violation complete, and complete.
In order not to favour a specific operational model, the hierarchy in Fig. 1 (middle) is cast in terms of abstract behavioural requirements for monitors. We then provide an instantiation that concretise those requirements into an operational hierarchy, establishing operational counterparts for each type of monitorability. To this end, we use the operational framework developed in [5]. We show this framework to be, in a suitable technical sense, maximally general (Thm. 4.1). This shows that our work is equally applicable to other operational models.
In order for a tool to synthesise monitors from specifications, it is useful to have syntactic characterisations of the properties that are monitorable with the required guarantees: synthesis can then directly operate on the syntactic fragment. We offer monitorability characterisations as fragments of recHML [32, 6] (a variant of the modal -calculus [29]) interpreted over finfinite traces—see Fig. 1 (right). The logic is expressive enough to capture all regular properties—the focus of nearly all existing definitions of monitorability—and subsumes more user-friendly specification logics such as LTL. Partial and complete monitorability already enjoy monitor synthesis functions and neat syntactic characterisations in recHML [5]; related synthesis functions based on syntactic characterisations for a branching-time setting [26, 27] have already been implemented in a tool [10, 9]. Here, we provide the missing syntactic characterisation for informative monitorability, and for a fragment of persistently informative monitorability.
Finally, we show that the proposed hierarchy accounts for existing notions of monitorability. See Fig. 1 (left). Safety, co-safety and their union correspond to partial monitorability and its two components, satisfaction- and violation-monitorability; Pnueli and Zaks’s definition of monitorability can be interpretated in two ways, of which one (pz) maps to informative monitorability, and the other (pz) to persistently informative monitorability. We also show that the definitions of monitorability proposed by Falcone et al. [24], contrary to their claim, do not coincide with safety and co-safety properties. To summarise, our principal contributions are:
A unified operational perspective on existing notions of monitorability, clarifying what operational guarantees each provides, see Thms. 3.1, 6.1 and 7.1; 2. 2.
An extension to the syntactic characterisations of monitorable classes from [5], mapping all but one of these classes to fragments in recHML, which can be viewed as a target byte-code for higher-level logics, see Thms. 5.2 and 5.3.
Proofs omitted from the body of this paper can be found in Appendix 0.C.
2 Preliminaries
Traces. We assume a finite set of actions, . The metavariables range over infinite sequences of actions. Finite traces, denoted as , represent finite prefixes of system runs. Collectively, finite and infinite traces are called finfinite traces. We use (resp., ) to range over finfinite traces (resp., sets of finfinite traces). A (finfinite) trace with action at its head is denoted as . Similarly, a (finfinite) trace with a prefix and continuation is denoted as . We write to denote that the finite trace is a prefix of , i.e., .
Properties. A property over finfinite (resp., infinite) traces is a subset of (resp., of ). Simply property refers to a finfinite property, unless stated otherwise. We say that a finite trace positively (resp., negatively) determines a property when (resp., ), for every . The same terms apply similarly when . We call a property regular if it is the union of a regular finite property and an -regular infinite property .
3 A Monitor-Oriented Hierarchy
From a tool-construction perspective, it is important to give concrete, implementable definitions of monitors; we do so in Sec. 4. To understand the guarantees that these monitors will provide, we first discuss the general notion of monitor and monitoring system. Already in this general setting, we are able to identify the various requirements that give rise to the hierarchy of monitorability, depicted in the middle part of Fig. 1. Sec. 4 will then provide operational semantics to this hierarchy, in the setting of regular properties.
We consider a monitor to be an entity that analyses finite traces and (at the very least) identifies a set of finfinite traces that it accepts and a set of finfinite traces that it rejects. We consider two postulates. Firstly, an acceptance or rejection verdict has to be based on a finite prefix of a trace, Def. 1.1; secondly, verdicts must be irrevocable, Def. 1.2. These postulates make explicit two features shared by most monitorability definitions in the literature.
Definition 1.
A monitoring system is a triple , where is a nonempty set of monitors, , and for every :
\bigl{(}\,\textbf{acc}(m,f)\text{ implies }\exists s\cdot\bigl{(}s\preceq f\text{ and }\textbf{acc}(m,s)\bigr{)}\,\bigr{)} and \bigl{(}\,\textbf{rej}(m,f)\text{ implies }\exists s\cdot\bigl{(}s\preceq f\text{ and }\textbf{rej}(m,s)\bigr{)}\,\bigr{)}; 2. 2.
\bigl{(}\textbf{acc}(m,s)\text{ implies }\forall f{\cdot}\textbf{acc}(m,sf)\bigr{)} and \bigl{(}\textbf{rej}(m,s)\text{ implies }\forall f{\cdot}\textbf{rej}(m,sf)\bigr{)}.
Remark 1.
Finite automata do not satisfy the requirements of Def. 1 since their judgement can be revoked. Standard Büchi automata are not good candidates either, since they need to read the entire infinite trace to accept or reject.
We define a notion of maximal monitoring system for a collection of properties; for each property in that set, such a system must contain a monitor that reaches a verdict for all traces that have some prefix that determines .
Definition 2.
A monitoring system is maximal for a collection of properties if for every there is a monitor such that iff trace has a prefix that positively determines ; iff trace has a prefix that negatively determines .
In Sec. 4, we present an instance of such a maximal monitoring system for regular properties. This shows that, for regular properties at least, the maximality of a monitoring system is a reasonable requirement. Unless otherwise stated, we assume a fixed maximal monitoring system throughout the rest of the paper. For to monitor for a property , it needs to satisfy some requirements. The most important such requirement is soundness.
Definition 3 (Soundness).
Monitor is sound for property when implies , and implies .
Lemma 1.
If is sound for and (resp., ), then positively (resp., negatively) determines .
Lemma 2.
For every : is sound for ; and if is a sound monitor for and (resp., ), then it is also the case that (resp., ).
The dual requirement to soundness, i.e., completeness, entails that the monitor detects all violating and satisfying traces. Unfortunately, this is only possible for trivial properties in the finfinite111In the infinite domain more properties are completely monitorable, see Sec. 8. domain—see Prop. 1. Instead, monitors may be required to accept all satisfying traces, or reject all violating traces.
Definition 4 (Completeness).
Monitor is satisfaction-complete for if implies and violation-complete for if implies . It is complete for if it is both satisfaction- and violation-complete for and partially-complete if it is either satisfaction- or violation-complete.
Proposition 1.
If is sound and complete for then or .
Proof.
If , then , so from Def. 1, . Due to the soundness of , . Similarly, when . ∎
We define monitorability in terms of the guarantees that the monitors are expected to give. Soundness is not negotiable. Given the consequences of requiring completeness, as evidenced by Prop. 1, we consider weaker forms of completeness. The weaker the completeness guarantee, the more properties can be monitored.
Definition 5 (Complete Monitorability).
Property is completely monitorable when there is a monitor that is sound and complete for . It is monitorable for satisfactions (resp., violations) when there is a monitor that is sound and satisfaction- (resp., and violation-) complete for . It is partially monitorable when it is monitorable for satisfactions or violations.
A class of properties is satisfaction-, violation-, partially, or completely monitorable, when every property is, respectively, satisfaction-, violation, partially or completely monitorable. We denote the class of all satisfaction, violation, partially, and completely monitorable properties by maximal monitoring systems as SCmp, VCmp, PCmp, and Cmp, respectively.
Since even partial monitorability, the weakest form in Def. 5, renders a substantial number of properties unmonitorable [5], one may consider even weaker forms of completeness that only flag a subset of satisfying (or violating) traces. Sound denotes monitorability without completeness requirements. Arguably, however, the weakest guarantee for a sound monitor of a property to be of use is if it flags at least one trace. One may then further strengthen this requirement and demand that this guarantee is invariant throughout the analysis of a monitor.
Definition 6 (Informative Monitors222These are not related to the informative prefixes from [30] nor persistence from [37]).
Monitor is satisfaction- (resp., violation-) informative if (resp., ). It is satisfaction- (resp., violation-) persistently informative if (resp., ). We simply say that is informative (resp., persistently informative) when we do not distinguish between satisfactions or violations.
Definition 7 (Informative Monitorability).
Property is informatively (resp., persistently informatively) monitorable if there is an informative (resp., a persistently informative) monitor that is sound for . A class of properties is informatively (resp., persistently informatively) monitorable, when all its properties are informatively (resp., persistently informatively) monitorable. The class of all informatively (resp., persistently informatively) monitorable properties by maximal monitoring systems is denoted as ICmp (resp., PICmp). A property is persistently informatively monitorable for satisfaction (resp., for violation) if there is a satisfaction- (resp., violation-) persistently informative monitor that is sound for . We revisit this definition in Sec. 4.
Example 2.
The property “f never occurs and eventually s is reached” (Example 1) is not partially monitorable but is persistently informatively monitorable.
The property requiring that “r only appears a finite number of times” is not informatively monitorable. For if it were, the respective sound informative monitor should at least accept or reject one trace. If it accepts a trace , by Def. 1, it must accept some prefix . Again, by Def. 1, all continuations, including , must be accepted by . This makes it unsound, which is a contradiction. Similarly, if rejects some , it must reject some finite that necessarily contains a finite number of r actions, making it unsound.
Theorem 3.1 (Monitorability Hierarchy).
The monitorability classes given in Defs. 5 and 7 form the inclusion hierarchy depicted in Fig. 1.
Proof.
The hardest inclusion to show is . Pick a property . Let . If then by Def. 4 we have . Otherwise, , meaning that positively determines , and by Def. 2 we have . By Def. 6, we deduce that is persistently informative since . Thus, by Def. 7, it follows that . The case for is dual. ∎
4 An Instantiation for Regular Properties
We provide a concrete maximal monitoring system for regular properties . This monitoring system that gives an operational interpretation to the levels of the monitorability hierarchy, and enables us to find syntactic characterisations for them in recHML [32, 5]. Since this logic is a reformulation of the modal -calculus [29], it is expressive enough to describe all regular properties and to embed specification formalisms such as LTL, (-)regular expressions, Büchi automata, and Street automata, used in the state of the art on monitorability.
The Logic.
The syntax or recHML is defined by the grammar in Fig. 2, which assumes a countable set of logical variables . Apart from the standard constructs for truth, falsehood, conjunction and disjunction, the logic is equipped with existential () and universal () modal operators, and two recursion operators expressing least and greatest fixpoints (resp., and ). The semantics is given by the function defined in Fig. 2. It maps a (possibly open) formula to a set of (finfinite) traces [5] by induction on the formula structure, using valuations that map logical variables to sets of traces, , where is the set of traces assumed to satisfy . An existential modality denotes all traces with a prefix action and a continuation that satisfies whereas a universal modality denotes all traces that are either not prefixed by or have a continuation satisfying . The sets of traces satisfying the least and greatest fixpoint formulae, and , are the least and the greatest fixpoints, respectively, of the function induced by the formula . For closed formulae, we use in lieu of (for some ). Formulae are generally assumed to be closed and guarded [31]. In the discussions we occasionally treat formulae, , as the properties they denote, .
Example 3.
The characteristic LTL operators can be encoded in recHML as:
[TABLE]
In examples, atomic propositions and resp., denote and .
For better readability, examples use LTL. Since we operate in the finfinite domain, X should be read as a strong next operator, in line with Example 3.
The Monitors.
We consider the operational monitoring system of [27, 5], summarised in Fig. 3 (symmetric rules for binary operators are omitted). The full system is given in Appendix 0.B. Monitors are states of a transition system where denotes an (external) choice and denotes a composite monitor where . There are three distinct verdict states, yes, no, and end, although only the first two are relevant to monitorability. This semantics gives an operational account of how a monitor in state incrementally analyses a sequence of actions to reach a new state ; the monitor accepts (resp., rejects) a trace , (resp., ), when it can transition to the verdict state yes (resp., no) while analysing a prefix . Since verdicts are irrevocable (rule mVer in Fig. 3), it is not hard to see that this operational framework satisfies the conditions for a monitoring system of Def. 1. The monitoring system of Fig. 3 is also maximal for regular properties, according to Def. 2. This concrete instance thus demonstrates the realisability of the abstract definitions in Sec. 3.
Theorem 4.1.
For all , there is a monitor that is sound for and accepts all finite traces that positively determine and rejects all finite traces that negatively determine .
As a corollary of Thm. 4.1, from Lem. 1 we deduce that for any arbitrary monitoring system , if is sound for some , then there is a monitor from Fig. 3 that accepts (resp., rejects) all traces that accepts (resp., rejects). In the sequel, we thus assume that the fixed monitoring system is of Fig. 3, as it subsumes all others.
5 A Syntactic Characterisation of Monitorability
We present syntactic characterizations for the various monitorability classes as fragments of recHML.
Partial Monitorability, syntactically.
In [5] Aceto et al. identify a maximal partially monitorable syntactic fragment of recHML.
Theorem 5.1 (Partial Monitorability [5]).
Consider the fragments:
[TABLE]
The fragment sHML is monitorable for violation whereas cHML is monitorable for satisfaction. Furthermore, if is monitorable for satisfaction (resp., for violation) by some , then it is expressible in cHML(resp., sHML) , i.e., (resp., ), such that .
As a corollary of Thm. 5.1, any that is monitorable for satisfaction (resp., for violation) can also be expressed as some (resp., ) where . For this fragment, the following automated synthesis function, which is readily implementable, is given in [5].
[TABLE]
Informative Monitorability, syntactically.
We proceed to identify syntactic fragments of recHML that correspond to informative monitorability.
Definition 8.
The informative fragment is where
[TABLE]
Theorem 5.2.
For , is informatively monitorable if and only if there is some , such that .
Example 4.
from Example 2 (expressed here in LTL) is a siHML property, as can be written in sHML as . In contrast, cannot be written in iHML, as it is not informatively monitorable.
Remark 2.
In siHML and ciHML, describes an informative part of the formula, that is, a formula with at least one path to (or ), which indicates that the corresponding finite trace determines the property. Monitor synthesis from these fragments can use this part of the formula to synthesize a monitor that detects the finite traces that satisfy (violate) . The value of the synthesised monitor then depends on . It is therefore important to have techniques to extract some that will retain as much monitoring information as possible. This extraction is outside the scope of this paper and left as future work.
Persistently Informative Monitorability, syntactically.
We also give a syntactic characterization of the recHML properties that are persistently informatively monitorable for satisfaction or violation. As the requirements for persistently informative monitors are more subtle than for informative monitors, the fragments we present are equally more involved than those for informative monitorability.
Definition 9.
We define eHML, the explicit fragment of recHML:
[TABLE]
Example 5.
Formula is not explicit, but it can be rewritten as the explicit formula .
Roughly, the following definition captures whether and are reacheable from subformulae (where the binding of a variable is reachable from the variable).
Definition 10.
Given a closed sHML (resp., cHML) formula , we define for a subformula that it can refute (resp., verify) in 0 unfoldings, when (resp., ) appears in , and that it can refute (resp., verify) in unfoldings, when it can refute (resp., verify) in unfoldings, or appears in and is in the scope of a subformula (resp., ) that can refute (resp., verify) in unfoldings. We simply say that can refute (resp., verify) when it can refute (resp., verify) in unfoldings, for some .
Example 6.
For formula , subformula can refute in [math] unfoldings. In contrast, cannot refute in [math] unfoldings, but it can refute in 1, because appears in it and can refute in [math] unfoldings. Therefore, all subformulae of can refute.
We now define the fragments of recHML corresponding to recHML properties that are persistently informatively monitorable for satisfaction or violation.
Definition 11.
We define the fragment where:
[TABLE]
Theorem 5.3.
For , is persistently informatively monitorable for violation (resp., for satisfaction) if and only if there is some (resp., ), such that .
Remark 3.
To the best of our efforts, a syntactic characterisation of persistently informative monitorability would involve pairs of equivalent formulae with parts from sHML and cHML that together become, in some sense, explicit. We leave such a characterization as future work.
6 Safety and Co-safety
The classic (and perhaps the most intuitive) definition of monitorability consists of (some variation of) safety properties [7, 38, 39, 24, 5]. Nevertheless, there are subtleties associated with how exactly safety properties are defined—particularly over the finfinite domain—and how decidable they need to be to qualify as truly monitorable. For example, Kim and Viswanathan [39] argued that only recursively enumerable safety properties are monitorable (they restrict themselves to infinite, rather than finfinite traces). By and large, however, most works on monitorability restrict themselves to regular properties, as we do in Sec. 4.
We adopt the definition of safety that is intuitive for the context of RV: a property can be considered monitorable if its failures can be identified by a finite prefix. This is equivalent to Falcone et al.’s formal definition of safety properties[24, Def. 4] and work such as [7, 18] when restricted to infinite traces.
Definition 12 (Safety).
A property is a safety property if every has a prefix, that determines negatively. The class of safety properties is denoted as Safe in Fig. 1.
Pnueli and Zaks, and Falcone et al. (among others) argue that it makes sense to monitor both for violation and satisfaction. Hence, if safety is monitorable for violations, then the dual class, co-safety (a.k.a. guarantee [24], reachability [17]), is monitorable for satisfaction. That is, every trace that satisfies a co-safety property can be positively determined by a finite prefix.
Definition 13 (Co-safety).
A property is a co-safety property if every has prefix, , that determines positively. The class of safety properties is denoted as CoSafe, also represented in Fig. 1.
Example 7.
“Eventually s is reached”, i.e., F s, is a co-safety property whereas “f never occurs”, i.e., , is a safety property. The property “s occurs infinitely often”, i.e., G F s, is neither safety nor co-safety. The property only holds over infinite traces so it cannot be positively determined by a finite trace. Dually, there is no finite trace that determines that there cannot be an infinite number of s occurrences in a continuation of the trace.
Safety and Co-safety, operationally.
It should come as no surprise that safety and co-safety coincide with an equally natural operational definition. Here, we establish the correspondence with the denotational definition of safety (co-safety), completing three correspondences amongst the monitorability classes of Fig. 1
Theorem 6.1.
* and .*
Proof.
We treat the case for safety, as the case for co-safety is similar. If is a safety property, then for every , there is some finite prefix of that negatively determines . Therefore, is sound (Lem. 2) and violation-complete (Def. 2) for . The other direction follows from Lem. 5. ∎
Aceto et al. [5] already show the correspondence between violation (dually, satisfaction) monitorability over finfinite traces and properties expressible in sHML (dually, cHML). As a corollary of Thm. 6.1, we obtain a syntactic characterisation for the Safe and CoSafe monitorability classes.
Remark 4.
Falcone et al. [24, Def. 17, Thm. 3] propose definitions of monitorability over finfinite traces that are claimed to coincide with the classes Safe, CoSafe and their union. However, this claim is incorrect. The properties, “the trace is finite” and G F s from Example 7 are neither safety nor co-safety properties. On the other hand, they are monitorable according to the alternative monitorability definition given in [24, Def. 17]. If the results claimed in [24, Thm. 3] held true, this would contradict the fact that those properties are neither safety nor co-safety properties. See Appendix 0.A for further details.
7 Pnueli and Zaks
The work on monitorability due to Pnueli and Zaks [35] is often cited by the RV community [13]. The often overlooked particularity of their definitions is that they only define monitorability of a property with respect to a (finite) sequence.
Definition 14 ([35]).
Property is -monitorable, where , if there is some such that is positively or negatively determined by .
Example 8.
The property \bigl{(}\textsf{f}\land\textsf{F}\,\textsf{r}\bigr{)}\vee\bigl{(}\textsf{F}\,\textsf{G}\,\textsf{s}\bigr{)} is -monitorable for any finite trace that begins with f, i.e., , since it is determined by the extension . It is not -monitorable for finite traces that begin with an action other than f.
Monitorability over properties—rather than over property–sequence pairs—can then be defined by either quantifying universally or existentially over finite traces: a property is monitorable either if it is -monitorable for all , or for some . We address both definitions, which we call pz- and pz-monitorability respectively. pz-monitorability is the more standard interpretation: it appears for example in [24, 14] where it is attributed to Pnueli and Zaks. However, the original intent seems to align more with pz-monitorability: in [35], Pnueli and Zaks refer to a property as non-monitorable if it is not monitorable for any sequence. This interpretation coincides with weak monitorability used in [20].
Definition 15 (pz-monitorability).
A property is (universally Pnueli–Zaks) pz-monitorable if it is -monitorable for all finite traces . The class of all pz-monitorable properties is denoted PZ.
Definition 16 (pz-monitorability).
A property is (existentially Pnueli–Zaks) pz-monitorable if it is -monitorable for some finite trace , i.e., if it is -monitorable. The class of pz-monitorable properties is written PZ.
The apparently innocuous choice between existential and universal quantification leads to different monitorability classes PZ and PZ.
Example 9.
Consider the property “Either s occurs before f, or r happens infinitely often”, expressed in LTL fashion as \bigl{(}(\neg\textsf{f})\,\textsf{U}\,\textsf{s}\bigr{)}\vee\bigl{(}\textsf{G}\,\textsf{F}\,\textsf{r}\bigr{)}. This property is pz-monitorable because every finite trace positively determines the property. However, it is not pz-monitorable because no extension of the trace f positively or negatively determines that property. Indeed, all extensions of f violate the first disjunct and, as we argued in Example 7, there is no finite trace that determines the second conjunct positively or negatively.
From Defs. 15 and 16, it follows immediately that \textsf{\forallPZ}\subset\textsf{\existsPZ}.
Proposition 2.
All properties in are pz-monitorable.
Proof.
Let and pick a finite trace . If there is an such that then, by Def. 12, there exists that negatively determines , meaning that has an extension that negatively determines . Alternatively, if there is no such that , itself positively determines . Hence is -monitorable, for every , according to Def. 14. The case for is dual. ∎
Pnueli and Zaks, operationally.
pz-monitorability coincides with informative monitorability: pz-monitorable properties are those for which some monitor can reach a verdict on some finite trace. For similar reasons, pz-monitorability coincides with persistently informative monitorability. See Fig. 1.
Theorem 7.1.
\textsf{\existsPZ}=\textsf{ICmp}* and \textsf{\forallPZ}=\textsf{PICmp}.*
Proof.
Since the proofs of the two claims are analogous, we simply outline the one for \textsf{\forallPZ}=\textsf{PICmp}. Let P\in\textsf{\forallPZ} and pick a finite trace . By Lem. 2, is sound for . By Def. 6 we need to show that there exists an such that or . From Defs. 15 and 14 we know that there is a finite such that positively or negatively determines . By Def. 2 we know that or . Thus , which is the required result.
Conversely, assume , and pick a . By Defs. 15 and 14, we need to show that there is an extension of that positively or negatively determines . From Defs. 6 and 7, there exists a such that or . By Def. 1, there is a finite extension of , say , that is a prefix of such that or . By Def. 2, we know that either positively or negatively determines . Thus P\in\textsf{\forallPZ}. ∎
8 Monitorability in other settings
We have shown how classical definitions of monitorability fit into our hierarchy and provided the corresponding operational interpretations and syntactic characterisations, focussing on regular finfinite properties over a finite alphabet and monitors with irrevocable verdicts. Here we discuss how different parameters, both within our setting and beyond, affect what is monitorable.
Monitorability with respect to the alphabet.
The monitorability of a property can depend on Act. For instance, if Act has at least two elements , property , which can be represented as , is -monitorable for every sequence , as can be extended to , which negatively determines the property. On the other hand, assume that . In this case, is neither pz- nor pz-monitorable. Indeed, no string , , determines positively or negatively as does not satisfy but its extension does. On the other hand, when restricted to infinite traces, is again pz-monitorable.
So far, we only considered finite alphabets; how an infinite alphabet, which may encode integer data for example, affects monitorability is left as future work.
Monitoring with revocable verdicts.
Early on, we postulated that verdicts are irrevocable. Although this is a typical (implicit) assumption in most work on monitorability, some authors have considered monitors that give revocable judgements when an irrevocable one is not appropriate. This approach is taken by Bauer et al. when they define a finite-trace semantics for LTL, called RV-LTL [14]. Falcone et al. [24] also have a definition of monitorability based on this idea (in addition to those discussed in Remark 4). It uses the four-valued domain ( for currently). Finite traces that do not determine a property yield a (revocable) verdict or that indicates whether the trace observed so far satisfies the property; yes and no are still irrevocable. This definition allows all finfinite properties to be monitored since it does not require verdicts to be irrevocable.
This type of monitoring does not give any guarantees beyond soundness: there are properties that are monitorable according to this definition for which no sound monitor ever reaches an irrevocable verdict: F G s for the system from Example 1 has no sound informative monitor, yet can be monitored according to Falcone et al.’s four-valued monitoring. This type of monitorability is complete, in the sense of providing at least a revocable verdict for all traces.
Monitorability in the infinite and finite.
Bauer et al. use pz-monitorability in their study of runtime verification for LTL [16] and attribute it to Pnueli and Zaks. However, unlike Falcone et al., Pnueli and Zaks [35] and ourselves, they focus on properties over infinite traces. There are some striking differences that arise if there is no risk of an execution ending. Aceto et al. show that, unlike in the finfinite domain, a set of non-trivial properties becomes completely monitorable: HML [28] (a.k.a. modal logic) is both satisfaction- and violation-monitorable over infinite traces [5]. Furthermore, some properties, like over , that were not pz- or pz-monitorable on the finfinite domain, are pz- or even pz-monitorable on the infinite domain. The full analysis of how the hierarchy in Fig. 1 changes for the infinite domain is left for future work.
Barringer et al. [12] consider monitoring of properties over finite traces. In this domain, all properties are monitorable if, as is the case in [12], the end of a trace is observable; in this setting the question of monitorability is less relevant.
9 Conclusion
We have proposed a unified, operational view on monitorability. This allows us to clearly state the implicit operational guarantees of existing definitions of monitorability. For instance, recall Example 1 from the introduction: since is pz- and pz-monitorable but it is not a safety nor co-safety property, we know there is a monitor which can recognise some violations and satisfactions of this property, but there is no monitor that can recognise all satisfactions nor all violations.
Although we focussed on the setting of regular, finfinite properties, the definitions of monitorability in Sec. 3, and, more fundamentally, the methodology that systematically puts the relationship between monitor behaviour and specification centre stage, are equally applicable to other settings. We have already mentioned the infinite domain and richer alphabets in Sec. 8. Another interesting direction would be to lift the restriction to regular properties and finite-state monitors. Indeed, while Barringer et al. consider a specification logic that allows for context-free properties [12], in [25], Ferrier et al. consider monitors with registers (i.e., infinite state monitors) to verify safety properties that are not regular. It seems likely that monitorability beyond safety may also be worth studying in these extended settings.
Acknowledgements
This research was partially supported by the projects “TheoFoMon: Theoretical Foundations for Monitorability” (grant number: 163406-051) and “Epistemic Logic for Distributed Runtime Monitoring” (grant number: 184940-051) of the Icelandic Research Fund, by the BMBF project “Aramis II” (project number:01IS160253) and the EPSRC project “Solving parity games in theory and practice” (project number:EP/P020909/1).
Appendix 0.A Monitorability à la Falcone et al.
Falcone et al. [24] propose three definitions of monitorability (Definitions 16 and 17 in [24]) which they claim to coincide with safety, co-safety, and the union of safety and co-safety properties (Theorem 3 in [24]). We discuss this claim in more detail here, and argue that it does not hold. In brief, their definition deems all properties that are uniform over finite traces, such as “success infinitely often”, or “the trace is finite” to be monitorable, not just safety and co-safety properties. In this appendix we recall Falcone et al.’s definitions and show that their definitions of monitorability include more than just safety and co-safety properties.
Remark 5.
Falcone et al. present finfinite properties as a pair consisting of a set of finite traces and a set of infinite traces. Here we will speak of just one set, containing both finite and infinite traces.
The definition of monitorability proposed by Falcone et al. in [24] is parameterised by a truth domain, and a mapping of formulas into this domain. They then give a uniform condition that defines monitorability with respect to any truth-domain and its associated mapping. Here we focus on their monitorability with respect to the truth-domains , and , which they claim correspond to co-safety, safety and their union, respectively.
Definition 17 (Property evaluation with respect to a truth-domain [24]).
For each of three different verdict-domains and finfinite properties (“-properties” in their terminology), Falcone, Fernandez and Mournier define the following evaluation functions:
** For and :**
if
otherwise.
** For and :**
if
otherwise.
For and :
if and
if and
otherwise.
Definition 18 (FFM-monitorability Definition 17, [24]).
A property is -monitorable over a truth domain if for all , if and , then .
From this definition, it easily follows that any property for which or is vacuously monitorable for any truth-domain, and evaluation function. However, not all such properties are safety or co-safety properties: “always eventually success” for instance is neither a safety nor a co-safety property.
We believe the critical points are Lemma 3 and Theorem 3 in [24], which do not hold. The proof of Lemma 3 in particular (Appendix 2.3) falsely claims that or implies that is a safety or co-safety properties.
Appendix 0.B An Operational Monitoring System: Regular Monitors
The monitoring system is given by a Labelled Transition System (LTS) based on Act, which is comprised of the monitor states, or monitors, and a transition relation. The set of monitor states, Mon, and the monitor transition relation, , are defined in Fig. 3. The suggestive notation denotes ; we also write to denote . We employ the usual notation for weak transitions and write in lieu of and for . We write sequences of transitions as , where . A monitor that does not use any parallel operator is called a regular monitor. The full monitoring system and regular monitors were defined and used in [1, 27, 5].
Definition 19 (Acceptance and Rejection).
For a monitor , we define (resp., ) and say that rejects (resp., accepts) when (resp., ). Similarly, for , we write (resp., ) if there exist and such that and rejects (resp., accepts) .
For a finite nonempty set of indices , we use to denote any combination of the monitors in using the operator . For each , is called a sum of , and is called a summand of . The following Lem. 3 assures us that regular monitors satisfy the conditions to be a monitoring system, given in Def. 1.
Lemma 3 (Verdict Persistence, [27, 5]).
We will use the following definitions and results in the proofs of Appendix 0.C. We define determinism for regular monitors.
Definition 20 ([4, 3]).
A closed regular monitor is * deterministic* iff every sum of at least two summands that appears in is of the form , where .
Definition 21 (Verdict Equivalence).
Monitors and are called verdict equivalent when for every , iff and iff .
Theorem 0.B.1 ([4, 5]).
Every monitor in Mon is verdict equivalent to a deterministic regular monitor.
In the following, we say that is suffix-closed when for all , implies — notice that we only quantify over finite traces. The suffix-closure of is .
Theorem 0.B.2 ([4, 3]).
If are regular and suffix-closed, and , then there is a regular monitor , such that iff and iff .
In this appendix, we use the formula synthesis function from regular monitors to sHML formulae, from [27, 5]:
[TABLE]
Theorem 0.B.3 ([5]).
For every regular monitor , , and is sound and violation-complete for .
Appendix 0.C Proofs Omitted from the Main Document
Here we present the proofs of results that were omitted from the main text.
We use the following classical result (see [8] for more on the -calculus and recHML):
Lemma 4.
If , then is regular.
Due to Thm. 0.B.1, we can assume that every monitor in Mon is a regular, or deterministic regular monitor. We often do so in the following proofs.
Definition 22.
Let . We define the following two sets of finite traces:
;
.
Lemma 5.
If is monitorable for satisfaction (resp., for violation) by any monitoring system, then every (resp., ) has a finite prefix that positively (resp., negatively) determines .
Proof.
We treat the case for satisfaction, as the case for violation is dual. Let and be a monitor that is sound and satisfaction-complete for . Then, due to satisfaction-completeness, , and by the requirements of Def. 1, there is a finite prefix of , such that . Therefore, by the same requirements, for every , . As we know that is sound for , this yields that positively determines . ∎
For a set of finite traces , we define
[TABLE]
Lemma 6.
Assume that every finfinite trace that satisfies (resp., violates) has a prefix that positively (resp., negatively) determines . Then (resp., ) is the suffix-closure of (resp., of ).
Proof.
Again, we only treat the case for satisfaction. Let . By our assumptions, there is at least one finite prefix of that positively determines . These prefixes of are well-ordered by the prefix relation, and therefore there is a smallest prefix of that positively determines . Therefore, , and we see that is in the suffix-closure of . Conversely, let and be an extension of . Then, , so by the proviso of the lemma, there is a prefix of (and of ) that positively determines ,333The reader may also notice that , due to the minimality of . and therefore . ∎
0.C.0.1 Proofs Omitted from Sec. 4: Regularity and Monitors
Lemma 7.
If , then and are regular.
Proof.
We know that is regular (Lem. 4) and , the infinite-trace interpretation of , is -regular. Therefore, there are a DFA that recognizes and a deterministic -automaton that recognizes . Let and . Let (resp., ) be the set of states in (resp., in ) that can be reached reading some trace . By construction, for each , we have that (resp., ) if and only if ends in the respective set in the automaton. Therefore, there are DFAs and for and , respectively, and thus is regular. The case for is similar. ∎
- Theorem 4.1
For all , there is a monitor that is sound for and accepts all finite traces that positively determine and rejects all finite traces that negatively determine .
Proof.
By Lem. 7, and , the sets of finite traces that (respectively) positively or negatively determine are regular. It is also not hard to see that they are suffix-closed. Therefore the theorem follows from Thm. 0.B.2. ∎
Corollary 1.
If is a sound monitor for , then there is a regular monitor that is sound for , and such that for every , implies , and implies .
Proof.
By Thm. 4.1, there is a regular monitor that is sound for , and accepts all finite traces that positively determine , and rejects all the finite traces that negatively determine . If (resp., ) for some finite trace , then, due to the soundness of , (resp., ), and therefore, from Lem. 1, positively (resp., negatively) determines . By the properties of , we have that (resp., ). ∎
0.C.0.2 Proofs Omitted from Sec. 5: Syntactic Characterizations
Definition 23.
Let be a closed regular monitor, and let be one of its submonitors. We say that can refute (resp., verify) in 0 unfoldings, when no (resp., yes) appears in , and that it can refute (resp., verify) in unfoldings, when it can refute (resp., verify) in unfoldings, or appears in and is in the scope of a submonitor of that can refute (resp., verify) in unfoldings. We simply say that can refute (resp., verify) in when it can refute (resp., verify) in unfoldings, for some .
We define the depth of in an sHML formula in a recursive way: ; ; ; ; and .
Lemma 8.
For all possibly open .
Proof.
Straightforward induction on . ∎
Lemma 9.
If , then there is a regular monitor that is sound and informative for .
Proof.
We assume that , as the case for is similar. Let , where and appears in . We prove by induction on that for all , if , then there is a finite trace that negatively determines . If , then , and we are done, as negatively determines . Otherwise, and we consider the following cases:
In this case, either or , so by the inductive hypothesis, there is a finite trace that negatively determines one of the two conjuncts, and therefore also .
In this case, , so, by the inductive hypothesis, there is a finite trace that negatively determines , so negatively determines .
In this case, . Therefore, from Lem. 8, , so, by the inductive hypothesis, there is a finite trace that negatively determines , so it also negatively determines , because .
As appears in , , so there is a finite trace that negatively determines , and therefore also . The theorem follows from Thm. 4.1. ∎
Lemma 10.
If and there is a monitor that is sound and informative for , then there is some such that .
Proof.
If is sound and informative for , then by Lem. 1, there is a finite trace that positively or negatively determines . Without loss of generality, we assume that positively determines . We can then easily construct a formula that is satisfied exactly by and all its extensions, recursively on : let , and let . Then, let . Thus, and . ∎
- Theorem 5.2
For , is informatively monitorable if and only if there is some , such that .
Proof.
A consequence of Lems. 9 and 10. ∎
Lemma 11.
If can refute in , then it is also the case that can refute in .
Lemma 12.
If all subformulae of or or can refute, then all subformulae of can refute. If all subformulae of can refute, then all subformulae of can refute.
We define the box-depth of a formula from recursively:
[TABLE]
Lemma 13.
For all possibly open .
Proof.
Straightforward induction on . ∎
Lemma 14.
Let and (resp., ), where all subformulae of can refute (resp., verify). There is some (resp., ), such that all subformulae of can refute (resp., verify), and for every , implies that (resp., implies that ).
Proof.
We assume that , as the case for is similar. Since is a closed formula and can refute, appears in , and therefore . We proceed to prove the lemma by induction on , similarly to the proof of Lem. 9.
If ,
then we are done immediately by taking .
If ,
then we can set .
If ,
then either or , and we are done by the inductive hypothesis on one of the two subformulae.
If ,
then and all subformulae of can refute, by Lem. 12. Furthermore, , and we are done by the inductive hypothesis. ∎
Lemma 15.
If or , then there is a regular monitor that is sound for and persistently rejecting, or, respectively, persistently verifying.
Proof.
We assume that , as the case for is similar. Let , where and all of its subformulae can refute. By Thm. 4.1, it suffices to prove that for every , there is some , such that negatively determines . We prove this by induction on . If , then as in the proof of Lem. 9, we can show that there is a finite trace that negatively determines . If , then by Lem. 14. there is some , such that all subformulae of can refute, and for every , implies that . By the inductive hypothesis, there is some , such that negatively determines , and therefore, negatively determines . ∎
We define the depth of a variable in a regular monitor recursively: and , where ; ; ; and .
Lemma 16.
Let be a persistently rejecting, deterministic regular monitor. If , then can only appear in as a submonitor of a larger sum.
Proof.
Let and let be an open monitor and a variable that does not appear in , such that . It is clear that . Therefore, it suffices to prove that for every deterministic with free variable , if , then there is a finite trace , such that there is no regular monitor for which . We proceed to prove this claim by induction on , and the case for is immediate. If , then, as is deterministic, , where , and we are done by the inductive hypothesis on either or , and . If , then if the inductive hypothesis on and gives trace , then we can set . If , then we re done by the inductive hypothesis on and . ∎
Here we call a regular monitor explicit when it is generated by the grammar:
[TABLE]
Corollary 2.
Every persistently rejecting, deterministic regular monitor is explicit.
Proof.
From Lem. 16. ∎
Lemma 17.
Let be an explicit deterministic regular monitor, such that all of its submonitors can refute. Then, and all of its subformulae can refute.
Proof.
By induction on the construction of . ∎
Lemma 18.
If and there is a monitor that is sound for and persistently rejecting or persistently verifying, then there is some , or, respectively, , such that .
Proof.
We treat the case where the monitor is persistently rejecting, as the case for a persistently verifying monitor is similar. From Cor. 1, there is a regular monitor, , that is sound for and persistently rejecting. By Thm. 0.B.1, we can assume that is deterministic (Def. 20). From Cor. 2, is explicit. If there is a submonitor of that cannot refute, then we can inductively on prove that there is a finite trace , for which there is no finite trace , such that , which is a contradiction. Therefore, from Lem. 17, the sHML formula that we can synthesize from is in eHML, and all of its subformulae can refute. Since is sound for and sound and violation complete for , it is the case that , and therefore and . ∎
- Theorem 5.3
For , is persistently informatively monitorable for violation (resp., for satisfaction) if and only if there is some (resp., ), such that .
Proof.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Luca Aceto, Antonis Achilleos, Adrian Francalanza, and Anna Ingólfsdóttir. Monitoring for silent actions. In Satya Lokam and R. Ramanujam, editors, FSTTCS , volume 93 of LIP Ics , pages 7:1–7:14, Dagstuhl, Germany, 2017. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik.
- 2[2] Luca Aceto, Antonis Achilleos, Adrian Francalanza, and Anna Ingólfsdóttir. A framework for parameterized monitorability. In Christel Baier and Ugo Dal Lago, editors, Foundations of Software Science and Computation Structures - 21st International Conference, FOSSACS 2018 , volume 10803 of Lecture Notes in Computer Science , pages 203–220. Springer, 2018.
- 3[3] Luca Aceto, Antonis Achilleos, Adrian Francalanza, Anna Ingólfsdóttir, and Sævar Örn Kjartansson. Determinizing monitors for HML with recursion. Co RR , abs/1611.10212, 2016.
- 4[4] Luca Aceto, Antonis Achilleos, Adrian Francalanza, Anna Ingólfsdóttir, and Sævar Örn Kjartansson. On the complexity of determinizing monitors. In Arnaud Carayol and Cyril Nicaud, editors, Implementation and Application of Automata - 22nd International Conference, CIAA 2017 , volume 10329 of Lecture Notes in Computer Science , pages 1–13. Springer, 2017.
- 5[5] Luca Aceto, Antonis Achilleos, Adrian Francalanza, Anna Ingólfsdóttir, and Karoliina Lehtinen. Adventures in monitorability: From branching to linear time and back again. Proceedings of the ACM on Programming Languages , 3(POPL):52:1–52:29, 2019.
- 6[6] Luca Aceto, Anna Ingólfsdóttir, Kim Guldstrand Larsen, and Jiri Srba. Reactive Systems: Modelling, Specification and Verification . Cambridge Univ. Press, New York, NY, USA, 2007.
- 7[7] Bowen Alpern and Fred B Schneider. Defining liveness. Information processing letters , 21(4):181–185, 1985.
- 8[8] André Arnold and Damian Niwinski. Rudiments of μ 𝜇 \mu -calculus , volume 146 of Studies in Logic and the Foundations of Mathematics . North-Holland, 2001.
