# Effectiveness Assessment of Cyber-Physical Systems

## TL;DR

This paper introduces a formal measure of effectiveness for Cyber-Physical Systems using measure theory, TBM, and Ev-IOHMM, enabling in vivo evaluation and benchmarking of system performance under uncertainties.

## Contribution

It develops a novel formal framework combining measure theory, Transferable Belief Model, and Ev-IOHMM to assess CPS effectiveness considering epistemic and aleatory uncertainties.

## Key findings

- The measure of effectiveness can evaluate autonomous vehicle controllers.
- The approach enables benchmarking against safety and well-being constraints.
- Application to autonomous vehicles demonstrates practical utility.

## Abstract

By achieving their purposes through interactions with the physical world, Cyber-Physical Systems (CPS) pose new challenges in terms of dependability. Indeed, the evolution of the physical systems they control with transducers can be affected by surrounding physical processes over which they have no control and which may potentially hamper the achievement of their purposes. While it is illusory to hope for a comprehensive model of the physical environment at design time to anticipate and remove faults that may occur once these systems are deployed, it becomes necessary to evaluate their degree of effectiveness in vivo. In this paper, the degree of effectiveness is formally defined and generalized in the context of the measure theory. The measure is developed in the context of the Transferable Belief Model (TBM), an elaboration on the Dempster-Shafer Theory (DST) of evidence so as to handle epistemic and aleatory uncertainties respectively pertaining the users' expectations and the natural variability of the physical environment. The TBM is used in conjunction with the Input/Output Hidden Markov Modeling framework (we denote by Ev-IOHMM) to specify the expected evolution of the physical system controlled by the CPS and the tolerances towards uncertainties. The measure of effectiveness is then obtained from the forward algorithm, leveraging the conflict entailed by the successive combinations of the beliefs obtained from observations of the physical system and the beliefs corresponding to its expected evolution. The proposed approach is applied to autonomous vehicles and show how the degree of effectiveness can be used for bench-marking their controller relative to the highway code speed limitations and passengers' well-being constraints, both modeled through an Ev-IOHMM.