
TL;DR
This paper proposes a formal, axiomatic framework for quantifying the efficiency of network defenses, ensuring the measure aligns with natural criteria and extending it to multiple variables.
Contribution
It introduces a unique, axiomatic definition of defense efficiency and generalizes it to multiple input variables, providing a rigorous foundation for automated defense evaluation.
Findings
The proposed efficiency measure satisfies natural axiomatic requirements.
The measure is uniquely characterized by these axioms.
Two natural generalizations of the measure are compared.
Abstract
In order to automate actions, such as defences against network attacks, one needs to quantify their efficiency. This can subsequently be used in post-evaluation, learning, etc. In order to quantify the defence efficiency as a function of the impact of the defence and its total cost, we present several natural requirements from such a definition of efficiency and provide a natural definition that complies with these requirements. Next, we precisely characterize our definition of efficiency by the axiomatic approach; namely, we strengthen the original requirements from such a definition and prove that the given definition is the unique definition that satisfies those requirements. Finally, we generalize the definition to the case of any number of input variables in two natural ways, and compare these generalizations.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNetwork Security and Intrusion Detection · Information and Cyber Security · Smart Grid Security and Resilience
11institutetext: University of Amsterdam
Defence Efficiency
Gleb Polevoy
Abstract
In order to automate actions, such as defences against network attacks, one needs to quantify their efficiency. This can subsequently be used in post-evaluation, learning, etc. In order to quantify the defence efficiency as a function of the impact of the defence and its total cost, we present several natural requirements from such a definition of efficiency and provide a natural definition that complies with these requirements. Next, we precisely characterize our definition of efficiency by the axiomatic approach; namely, we strengthen the original requirements from such a definition and prove that the given definition is the unique definition that satisfies those requirements. Finally, we generalize the definition to the case of any number of input variables in two natural ways, and compare these generalizations.
1 Introduction
Exact definitions and measurements are necessary for conducting science [2, Page ]. In particular, many actions, such as defending against network attacks, defragmenting disks, cleaning streets, and locating viruses, would benefit from knowing their efficiency. Knowing efficiency would allow for improvements, including automatic improvements. We therefore need to come up with a proper definition of the efficiency of actions.
Since we are not aware of a general definition of efficiency, we define it in a natural way satisfying natural axioms, such as monotonicity with respect to certain inputs. This is a derived measure which inputs constitute a Cartesian product [2, Chapter ]. A famous problem of deriving measures is the arbitrary choice in the definition [1], and there is some work on the existence and uniqueness of a derived measurement of a certain form [2, Section ], but we would like to have stronger uniqueness statements than homomorphisms. Therefore, we ensure our definition is the best for our goals by axiomatic characterisation. We begin with an example of defending against a network attack, and then we suggest a general formula which we characterise by the means of some natural axioms. Next, we generalise this formula to any number of input parameters in two possible ways. The first way is an expansion of the basic formula, and we characterise this expansion by an expended set of axioms. The second generalisation is a combination of the basic formulas. We finally explore the relationships between these two natural generalisations.
To summarise, we suggest a natural uniquely characterisable way to define efficiency and study its generalisations, allowing for a wide range of applications.
2 Network Defence
Let us consider the scenario of defending against an attack in the a network. Consider a network where a node is under attack, which reduces its revenue. The implemented countermeasure can either bring upon a recovery or not.
Denote the revenue as a function of time be , and assume that . Let the detection of the attack be and the recovery from it be ; and let the baseline level be . Then, the impact of an attack is defined as . If no recovery takes place, then , and then, . This improper integral can either converge or not.
Let the cost of implementing a countermeasure be , assuming that . Then, we define the total cost as . As the case with revenue, if no recovery is achieved, this integral is improper and it can either converge or not.
In practice, we measure the revenue and the costs of the countermeasure for a given time bound ; in particular, all the integrals are taken at most till . If no recovery takes place within this period, we call the case not to have recovered and take the integral till , instead of infinity. This boundedness of time allows us work with only proper integrals (the revenue and cost functions are assumed to be bounded anyhow). Thus, the impact and the total cost are always finite, regardless whether we consider a recovery to take place.
3 Single Revenue and Cost
We now model the situation and define the efficiency of a countermeasure for the scenario above, though this can be applied to infinitely many practical situations, where the efficiency decreases in both numerical inputs.
Let be an upper bound on the cost during the period . Preparing to define the efficiency of a countermeasure, and we require it to have the following properties:
Monotonously decreasing with impact , where . 2. 2.
Monotonously decreasing with total cost , where . 3. 3.
If no recovery takes place, the efficiency is always smaller than if a recovery does take place, regardless anything else. 4. 4.
All the values between [math] and are obtained, and only they are. In the functional notation, efficiency is a function .
From the infinitely many definitions of efficiency that fulfill all the above properties, we propose the following one. We define the efficiency as
[TABLE]
where parameter defines the division point between recovery and no recovery (we allocate of the total scale to the case of no recovery, and the rest is given to the case of recovery), and parameter expresses the relative importance of the impact w.r.t. the total cost. The idea is to combine the relative saved revenue with the relative saved cost , and shift the recovered case in front of the non-recovered one. The multiplication by normalizes the efficiency of no recovery to fit to .
The expression can obtain all the values in , as is in . The expression obtains all the values in , as . Therefore, the defined efficiency obtains the values in if a recovery takes place, and the values in otherwise. The continuity of the efficiency function implies that all the values in these segments are obtained.
Practically, we should take to be the smallest known upper bound, because a non-tight bound makes the efficiency seem larger, since will then never become zero, even for a very costly countermeasure.
The following characterization theorem proves that Eq. (1) is the unique definition of efficiency, if we require a stronger set of properties.
Theorem 3.1
Let obtain values in . Then, Eq. (1) is the unique definition of efficiency that satisfies the following set of properties:
Linearly decreasing with impact , where . 2. 2.
Linearly decreasing with total cost , where . 3. 3.
The ratio of the linear coefficient of the impact to the linear coefficient of the total cost is the same, regardless whether the recovery takes place or not. 4. 4.
If no recovery takes place, all the values between [math] and and only they can be obtained; if a recovery does take place, then all the values between and and only they can be obtained.
We remark that condition 3 implies that the ratio of the linear coefficient of the impact to the linear coefficient of the total cost expresses their relative importance, regardless whether recovery takes place.
Proof
Eq. (1) is linearly decreasing with impact and with total cost and condition 3 holds in a straight-forward manner. We have showed after the definition of Eq. (1) that condition 4 is fulfilled as well. It remains to prove the other direction.
Let the formula for the case when a recovery is attained be , for positive and . This form follows from conditions 1 and 2. For the minimum impact and total cost, , we have the maximum possible efficiency of , implying that . For the maximum impact and total cost, and , we have the minimum possible efficiency of , which means that . Let be . The nonnegativity of and imply together that , as required from in Eq. (1). Moreover, implies that . To conclude, the efficiency is , where and , for , as in Eq. (1).
In the case of no recovery, let the formula be . By substituting we conclude that . By substituting and , we obtain , i.e. . From condition 3 we have
[TABLE]
These two equations, together with the proven above equality , imply that each coefficient gets multiplied by , yielding and . Together with the expression above for , we obtain Eq. (1).
4 Generalizations
The work till now assumed two inputs to the efficiency, besides the fact whether the system has recovered: the impact and the total cost. However, in some cases, more input variables are relevant. For instance, consider the case with multiple revenues. We denote the th revenue by , its baseline level by and the corresponding th impact , i.e. , where and are the th detection and recovery time, respectively. Further, denote the th cost of a countermeasure by , and the th total cost by , i.e. . We can also have various time bounds for the countermeasures for various revenues, and we say that the system has recovered if all the revenues have recovered.
In general, we may have various input variables of any nature, which should have positive or negative influence on the defined efficiency. We present two different natural generalizations of the work above to multiple variables. First, we expand Eq. (1) to consist of multiple terms. The second natural option is to simply combine equations of type Eq. (1).
4.1 Expanding the Equation
We generalize Eq. (1) as follows.
We allow the efficiency decrease in strictly increasing functions of possibly multiple factors. Such a factor , where the strictly increasing function is , such that , appears in the formula as , where is an upper bound on . This equation obtains all the values from to [math], when ranges from [math] to . 2. 2.
The efficiency may also increase in strictly increasing functions of additional, possible multiple, factors. Such a factor , where the strictly increasing function is , appears as , where is an upper bound on . This equation obtains all the values from [math] to , when ranges from [math] to .
The factors w.r.t. to increasing functions of which the the dependency is increasing are denoted as , and the factors w.r.t. to increasing functions of which the the dependency is decreasing are denoted as . Then, generalizes to , describing the importance of the given function of each factor, and the efficiency is defined as follows (w.l.o.g., we assume here that the efficiency is decreasing w.r.t. at least one factor).
[TABLE]
As before, defines the division point between recovery and no recovery. The parameters fulfill is between [math] and .
Analogously to the basic case (where and ), we can show that this efficiency fulfills the following conditions.
Monotonously increasing with each , where . 2. 2.
Monotonously decreasing with each , where . 3. 3.
If no recovery takes place, the efficiency is always smaller than if a recovery does take place, regardless anything else. 4. 4.
All the values between [math] and are obtained, and only they are. In the functional notation, efficiency is a function .
We generalize Theorem 3.1 as follows.
Theorem 4.1
For , let obtain values in , and for , let be in . Then, Eq. (2) is the unique definition of efficiency that satisfies the following set of properties:
Linearly increasing with each , where . 2. 2.
Linearly decreasing with each , where . 3. 3.
The ratio of the linear coefficient of the function of any variable to the linear coefficient of the function on any other variable is the same, regardless whether the recovery takes place or not. 4. 4.
If no recovery takes place, all the values between [math] and and only they can be obtained; if a recovery does take place, then all the values between and and only they can be obtained.
Proof
This theorem is proven analogously to Theorem 3.1, besides proving that the conditions of this theorem imply the formula also for the case of no recovery, after having proven the rest. We prove this part now. Conditions 1 and 2 allow us assume that the utility for no recovery looks as . First, the maximal possible value for no recovery, , is obtained by substituting for each respective and zeros for each . This substitution yields , implying that .
The least possible value for no recovery is zero, and it is attained when each is an each is zero. This provides , implying . Condition 3 implies that for any two variables, w.l.o.g., for and there holds111Analogously for and , or for and .
[TABLE]
where and are the coefficients for the case of recovery. Assuming we have proven the formula for the case of recovery, we know that . Since the ratios of the coefficients remain the same, but the sum in the case of no recovery is instead of , we need to multiply the coefficients of the case of recovery by . In particular, the above equation implies that , completing the proof.
We now remark on making this generalization even more general.
Remark 1
Each variable and can be equipped with its own increasing function, and this leaves the statements and their proofs unchanged.
4.2 Combining Equations
The generalization we present now allows for several inputs, as Section 4.1 does, and furthermore, we can now also allow for some revenues to have recovered while others may have not recovered.
We use the following efficiency as a black box to define the efficiency of the th countermeasure, .
[TABLE]
The total efficiency is then defined as follows:
[TABLE]
where the nonnegative parameter describes the importance of th revenue. By taking normalized s, such that the combination is convex, meaning that , we ensure that is in , because all the s are there.
Assuming that either all the revenues recover or not, as required by the previous generalization, a natural question is about the connection between this and the previous generalization of Eq. (1). In general, the two generalizations are not equivalent, as the following example demonstrates. The gist of this example is that linear combination treats the ratios in the recovery and the non-recovery cases differently.
Example 1
For , let the efficiency of the th countermeasure be
[TABLE]
for the case of recovery and no recovery, respectively. Then, in the combined efficiency of the two countermeasures, the ratio between the coefficients of and in the case of recovery is
[TABLE]
while in the case of no recovery, the ratio is
[TABLE]
These are, generally speaking, not equal: for instance, by substituting , , and , Eq. (5) gives , but Eq. (6) yields . These ratios are not equal, thereby violating condition 3 of Theorem 4.1. Therefore, expanding equations is not equivalent to combining equations.
However, when the system recovers, the two generalizations are equivalent, as we prove next.
Proposition 1
If all the system recovers, then expanding equations is equivalent to combining equations.
Proof
Combining equations includes expanding an equation, because any equation of the form Eq. (2) can be obtained by combining equations and with the coefficients .
In order to show that expanding includes combining, we prove that combining equations of the form Eq. (2), which includes Eq. (3), using Eq. (4) yet again yields an equation of the form Eq. (2). Consider the part of Eq. (2) that refers to the case when a recovery is achieved, and look at the expression before the equality sign. The expression is plus a linear combination of terms such as and , such that the sum of the coefficients of these terms is . Therefore, combining such equations, the th equation having , according to Eq. (4) will yield plus a linear combination of terms such as and , such that the sum of the coefficients of these terms is , which is exactly an expression of the type of Eq. (2) for the case of recovery.
5 Conclusion
We first presented a basic efficiency model where we had two parameters, characterised it axiomatically, and subsequently generalized it in two natural ways. Then, we compare these two ways, showing that they are generally not the same, but if a recovery takes place, then they are the same. Basically, the characterisations and the partial equivalence of the natural generalisations hint that there may be only one natural way to approach efficiency.
We may look at another axioms and at other generalisations. It would be nice to axiomatise the way the generalisation stands with respect to the original formula. We may also consider eliciting the paramters of the formulas. While using the formulas in practice, we may want to be able to recalculate the results after an update about the values of some parameters arrive. In order to be able to perform that without storing extra information, we may need to look for other formulas.
To conclude, we provide a natural definition of efficiency, the only definition that fulfils a natural set of axioms.
Acknowledgments
This research is funded by the Dutch Science Foundation project SARNET (grant no: CYBSEC.14.003 / 618.001.016)
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Adams, E.W.: On the nature and purpose of measurement. Synthese 16(2), 125–169 (Nov 1966)
- 2[2] Roberts, F.S.: Measurement Theory: With Applications to Decisionmaking, Utility, and the Social Sciences, Encyclopedia of Mathematics and its Applications, vol. 7. Cambridge University Press (1984)
