Rethinking System Health Management

TL;DR

This paper advocates for unifying system health management with decision making in complex systems, leveraging recent advances to improve operational effectiveness and reduce complexity.

Contribution

It introduces a unified formulation for system health management and decision making, addressing limitations of traditional separate approaches.

Findings

Unified approach increases operational effectiveness.

Numerical examples illustrate limitations of traditional methods.

Proposed method accommodates typical health management concepts.

Abstract

Health management of complex dynamic systems has traditionally evolved separately from automated control, planning, and scheduling (generally referred to in the paper as decision making). A goal of Integrated System Health Management has been to enable coordination between system health management and decision making, although successful practical implementations have remained limited. This paper proposes that, rather than being treated as connected, yet distinct entities, system health management and decision making should be unified in their formulations. Enabled by advances in modeling and computing, we argue that the unified approach will increase a system's operational effectiveness and may also lead to a lower overall system complexity. We overview the prevalent system health management methodology and illustrate its limitations through numerical examples. We then describe the proposed unification approach and show how it accommodates the typical system health management concepts.

Figures

Equations

$$\begin{array}{rl}& E[{\text{EOL}}_d(t_p)]=\frac{{\rho }_p}{{\dot{s}}_n}\text{ and }E[{\text{EOL}}_s(t_p)]=\frac{{\rho }_p}{p_n{\dot{s}}_n+p_h{\dot{s}}_h}=\\ & =\frac{{\rho }_p}{(1-p_h){\dot{s}}_n+p_h({\dot{s}}_n+ϵ/\Delta t)}=\frac{{\rho }_p}{({\dot{s}}_n+p_hϵ/\Delta t))},\text{ then}\end{array}$$

$$\sigma =\left|\frac{{\rho }_p}{{\dot{s}}_n}-\frac{{\rho }_p}{({\dot{s}}_n+p_hϵ/\Delta t)}\right|={\rho }_p\left|20-\frac{1}{(0.05+p_hϵ)}\right|\Delta t.$$

$$U(s,a)=\sum _{s^{\prime }}T(s^{\prime },a,s)U(s^{\prime }).$$

$$U^{a_{0:H}}(s)=\sum _{i=0:H}R(s_i,a_i).$$

$$U_t^{\pi }(s)=R(s,\pi (s))+\gamma \sum _{s^{\prime }}T(s,\pi (s),s^{\prime })U_{t-1}^{\pi }(s^{\prime }),$$

$$U_t^{\ast }(s)=\underset{a\in A}{\max }(R(s,a)+\gamma \sum _{s^{\prime }}T(s,a,s^{\prime })U_{t-1}^{\ast }(s^{\prime })),$$

$${\pi }^{\ast }(s)=\underset{a\in A}{\mathrm{argmax}}(R(s,a)+\gamma \sum _{s^{\prime }}T(s,a,s^{\prime })U^{\ast }(s^{\prime })).$$

Full text

1 Introduction

System Health Management (SHM) has evolved from simple red-line alarms and human-initiated responses to a discipline that often includes sophisticated modeling and automated fault recovery recommendations (?). The main goal of modern SHM has been defined as the preservation of a system’s ability to function as intended (?; ?). While in this paper we apply the term SHM to the operational phase of a system’s lifespan, in other contexts the term may also encompass design-time considerations (?). The actual achievement of a system’s operational objectives, on the other hand, is under the purview of the fields of control, planning, and scheduling. In this paper, we will generally refer to all of the processes aimed at accomplishing operational objectives as decision making (DM), while using the more specialized terms where necessary.

Historically, the field of SHM has developed separately from DM. The typical SHM functions (monitoring, fault detection, diagnosis, mitigation, and recovery) were originally handled by human operators–and similarly for DM (?; ?). As simple automated control was being introduced for DM, automated fault monitors and alarms reduced operator workload on the SHM side. Gradually, more sophisticated control techniques were developed for DM (?), while automated emergency responses started handling some of the off-nominal system health conditions (?). Capable automated planning and scheduling tools eventually became available for performing more strategic DM (?). On the SHM side, automated fault diagnosis was added, in some cases coupled with failure prediction (i.e., prognostic) algorithms (?), as well as with recovery procedures (?). Still, the two sides largely remained separated. When the concept of Integrated System Health Management became formalized, most interpretations of integrated encompassed some interactions between DM and SHM, although practical implementations of such interactions have been limited (?).

This paper makes the following claims:

For actively controlled systems, prognostics is not meaningful; for uncontrolled systems prognostics may only be meaningful under a specific set of conditions; 2. 2.

DM should be unified with SHM for optimality and there is a path for doing so; 3. 3.

Automated emergency response should be done separately from unified DM/SHM, in order to provide performance guarantees and dissimilar redundancy.

For the second claim, we intend to show a path towards DM/SHM unification that builds on the latest developments in state space modeling, planning, and control. While in the past limitations in computing hardware and algorithms would have made unified DM/SHM difficult, we believe that the advances of the recent years make it an attainable goal. We also believe that this unified approach is applicable to a broad spectrum of DM, from traditional deterministic planners to complex algorithms computing long-horizon action policies in the presence of uncertainty. We use the term unification to emphasize the idea of DM and SHM being done within the same framework, rather than the two being integrated as separate subsystems exchanging information.

Some initial progress towards DM/SHM unification can be found in the earlier work by ? (?), ? (?), and ? (?; ?). System health information was also incorporated into DM by others (?; ?; ?). This paper aims to introduce a systematic view on such integration, discuss its benefits, and illustrate how current SHM concepts map into the proposed approach without a loss of functionality or generality.

The paper first defines the categories of systems that are of interest to this study (Section 2) and then overviews the prevailing approach to SHM (Section 3). The first claim (on prognostics) is discussed in Section 4. Section 5 discusses the second claim, concerning DM/SHM integration and its benefits, as well as the rationale for the third claim (that DM/SHM should be separate from automated emergency response). Section 6 concludes.

2 Systems of interest

In the discussion to follow, we consider both uncontrolled and controlled systems. For our purposes, the uncontrolled systems category includes not only those systems types for which control is not available or required, but also those operating on predefined control sequences, such as industrial robots performing the same sets of operations over extended periods of time. Also included are system types that can be considered uncontrolled within some time interval of interest (decision horizon). Systems in this category may have degradation modes that affect the system’s performance within its expected useful life span. The rate of degradation is influenced by internal (e.g., chemical decomposition) and external factors (e.g., temperature of the operating environment). In addition to aforementioned industrial robots, examples of uncontrolled system types include bridges, buildings, electronic components, and certain types of rotating machinery, such as electrical power generators.

The controlled systems category covers all other system types, including dynamically controlled systems operating in uncertain environments, where the current state cannot be fully observed and non-determinism is present in control action outcomes. Degradation processes are influenced not only by the same kinds of internal and external factors as for the uncontrolled systems, but also by the control actions.

Most of the discussion to follow is applicable to both categories, although controlled systems would, naturally, benefit more from active DM/SHM. In describing the systems, we adopt the notation from the field of decision making under uncertainty (?).

A system state $s$ can be a scalar or a vector belonging to a state space $S$. A system action $a$ initiates state transitions, with $A$ denoting the space of all available actions ($A$ may be state-dependent). A transition model $T(s,a,s^{\prime})$ describes the probability of transitioning to a particular state $s^{\prime}\in S$ as a result of taking action $a$ from state $s$. A reward model $R(s,a)$ describes a positive reward obtained or a negative cost incurred as a result of taking action $a$ from state $s$. Terminal states form a subset $S_{T}\subset S$. Terminal states $S_{T}$ may include both failure and goal states. Transitions from a terminal state are only allowed back to itself.

If there is state uncertainty, belief states are used instead of regular states (also referred to as beliefs). A belief $b$ is a probability distribution over $S$, with $B$ denoting the space of all beliefs. Observations (e.g., sensor readings) can help with belief estimation and updating. Like a state, an observation can be a vector quantity. An observation model $O(s^{\prime},a,o)$ describes the probability of getting an observation $o$ upon transition to state $s^{\prime}$ as a result of action $a$.

The general function of decision making is to select actions. While in some systems we are only concerned with selecting a single $a_{t}$ at a given time $t$, decision making problems often involve selecting a sequence of actions. In a strictly deterministic system, an entire sequence of actions (a plan) can be selected ahead of time. In systems with action outcome uncertainty, however, a fixed plan can quickly become obsolete. Instead, a policy $\pi(s):S\rightarrow A$ needs to be selected that prescribes which action should be taken in any state. If a policy is selected that, in expectation, optimizes a desired metric (e.g., maximizes cumulative reward), it is referred to as an optimal policy and denoted $\pi^{*}$.

Throughout the paper, we use a robotic exploration rover operating on the surface of the Moon as a running example of a complex controlled system. The rover is solar-powered and stores electrical energy in a rechargeable battery.

3 System Health Management

A typical contemporary SHM integration approach is shown in Figure 1. A DM subsystem generates an action $a_{t,\text{DM}}$, aimed at achieving the operational objectives. The plant executes $a_{t,\text{DM}}$ and an observation $o_{t}$ is generated and relayed to both DM and SHM. DM computes $a_{t+1,\text{DM}}$ on the basis of $o_{t}$, while SHM analyzes $o_{t}$ for indications of faults (defined here as system states considered to be off-nominal) and, if any are detected, issues a mitigation or recovery command $a_{t+1,\text{SHM}}$ either directly to the plant or as a recommendation to the DM subsystem (?).

Figure 2 goes into more detail on the SHM subsystem. The fault detection module corresponds to the traditional red-line monitors detecting threshold-crossing events of sensor values, represented on the diagram by a Boolean fault detection function $F$. If a fault is detected ($F(o_{t})=\text{true}$), fault isolation and diagnosis (or identification) is performed, generating a vector of fault descriptors $f_{t}$ (?). Each fault descriptor typically identifies a component, its fault mode, and its fault parameters (?). There are diagnostic systems that also include an estimated fault probability in the descriptor (?). If the uncertainty of the diagnostic results is deemed too high (e.g., $f_{t}$ consists of only low-probability elements), uncertainty management is sometimes performed in order to obtain a better estimate of the current system condition (?).

Some recent SHM implementations then pass $f_{t}$ to a prognostic module (?). In the SHM context, the intended goal of the prognostic module is to predict, at time $t_{p}$ (here $t_{p}=t$), whether and when faults will lead to system failure (defined as inability to perform its assigned function) within the window $[t_{p},t_{p}+H]$ of a prediction horizon $H$ (the terms prediction horizon and decision horizon are equivalent for our purposes). In prognostics literature, the time of failure is commonly synonymous with the term end of [useful] life (EOL). Equivalently, the goal of prognostics can be defined as predicting the system’s remaining useful life (RUL). In Figure 2, the prognostic prediction is represented as a probability distribution $p(\text{EOL}|f_{t})$ for $\text{EOL}\in[t_{p},t_{p}+H]$. Uncertainty management is sometimes also prescribed following prognostic analysis, meant to improve the prediction if the confidence in it is insufficiently high (?). Note that if a prognostic module is part of an SHM sequence, the term Prognostics and Health Management (PHM) is used by some instead of SHM in order to emphasize the role prognostics is playing in managing the system’s lifecycle.

Finally, $p(\text{EOL}|f_{t})$ and $f_{t}$ are passed to the fault mitigation and recovery component to select an action $a_{t+1,\text{SHM}}$ from the action set $A_{\text{SHM}}$, in order to mitigate or recover from faults in $f_{t}$. As part of this process, operational constraints may be set for those faulty components that cannot be restored to nominal health. If functional redundancy exists for such components, their further use may be avoided.

The overall limitations of the current SHM approach are discussed in Section 5, where an approach that unifies DM and SHM is then proposed. The next section, however, focuses on the prognostic component and discusses why it is not meaningful for actively controlled systems and is challenging to implement in a useful manner for uncontrolled systems.

4 Prognostics

A general definition of prognostics is that of a process predicting the time of occurrence of an event $E$ (?). Using notation from Section 2, if $\phi_{E}:S\rightarrow\mathbb{B}$ (where $\mathbb{B}\triangleq\{0,1\}$) is an event threshold function, then $t_{E}\triangleq\text{inf}\{t\in[t_{p},t_{p}+H]:\phi_{E}(s_{t})=1\}$ is the nearest predicted time of $E$ ($s_{t}$ is the state at time $t$). If the state evolution trajectory is non-deterministic, then $p(t_{E}|s_{0:t_{p}})$ is computed instead. If states cannot be directly observed, $p(t_{E}|o_{0:t_{p}})$ is computed. As defined, prognostics is only meaningful in a specific set of circumstances and next we use two examples to illustrate why.

4.1 Uncontrolled systems

There are two main desirable, interrelated attributes for a prognostic system: (1) low uncertainty in EOL estimation, so that a decision about mitigation or recovery actions can be made with confidence, and (2) the ability to make a prediction far enough in advance for the actions to be successfully executed. For uncontrolled systems, this means that prognostics is primarily useful for systems with long lifetimes, low process uncertainty, or both. To illustrate why this is the case, we start with a simple uncontrolled system example:

Example 1.

At time $t=0$, the health state of a system is $s_{0}=1$ (states are scalar). According to a deterministic model, the nominal health degradation rate is constant at $\dot{s}_{n}=0.05/\Delta t$, where $\Delta t$ is the prediction time step, selected as the minimum time interval within which a change in the system’s health is expected to be detectable. A stochastic model predicts the probability of the nominal degradation rate $\dot{s}_{n}$ within any time step as $p_{n}=0.8$ and the probability of a higher degradation rate ($\dot{s}_{h}=\dot{s}_{n}+\epsilon/\Delta t$) as $p_{h}=0.2$. Assume $\epsilon=0.05$.

The objective for both models is to predict EOL, i.e., the smallest $t$ for which $s\leq 0$. For this example, the prediction uncertainty is defined as $\sigma(t_{p})=|E[\text{EOL}_{d}(t_{p})]-E[\text{EOL}_{s}(t_{p})]|$, i.e., the absolute difference between the expected EOL values computed by the two models at a prediction time $t_{p}$. A requirement is set on the maximum EOL prediction uncertainty as $\sigma_{\textit{max}}=1\Delta t$.

For this example, let us assume that the health state is fully observable and define $\rho_{p}=s_{p}/s_{0}$, the fraction of full health remaining at $t_{p}$. In Figure 3, a prediction is shown to be made at $t_{p}=t_{0}$, with $s_{p}=s_{0}$, $\rho_{p}=1$, and $H=20\Delta t$. Since

[TABLE]

[TABLE]

In the last equation, we substitute the value for the nominal degradation rate $\dot{s}_{n}$ in order to focus on the effects of the degradation rate uncertainty. As can be seen in Figure 3, EOL is reached by both models within the prediction horizon. However, from Equation 1, $\sigma=3.33\Delta t>\sigma_{max}$.

For the requirement on $\sigma$ to be satisfied, either $\rho_{p}$ (health fraction remaining), $p_{h}$ (the probability of deviations from the nominal degradation rate), $\epsilon$ (the magnitude of deviations), or some combination of them needs to be reduced. If $p_{h}$ and $\epsilon$ are kept the same, with $\rho_{p}=0.25$ we can get $\sigma=0.83\Delta t$. However, $t_{p}$ now needs to be $\approx 15\Delta t$, with only $5\Delta t$ left until failure (RUL). RUL corresponds to the time available to either replace/repair the uncontrolled system or initiate an emergency response. For a quickly degrading system with $\Delta t=1\,\mathrm{s}$, RUL would be only $5\,\mathrm{s}$, which is likely enough time for an emergency response, but not for repair or replacement. In practice, in uncontrolled systems where handling a fast-developing degradation process is important, estimating $p(\text{EOL})$ is unlikely to bring tangible benefits. For instance, if pressure starts building quickly in a fuel tank of an ascending crewed rocket, the launch abort system (emergency response) is likely to be activated by the exceedance of a predefined pressure limit or pressure increase rate (i.e., functions of fault detection). Computing whether the tank breach will occur in $10$ seconds or $12$ seconds will not materially influence that response. For uncontrolled systems with mid-range degradation rates (minutes, hours, days), extrapolation of the degradation function may be able to serve as part of a caution and warning mechanism.

What follows is that if degradation uncertainty is relatively high or varies significantly over time, a short prediction horizon (compared to the overall system lifetime) may be necessary to limit the uncertainty propagation and result in a usable $\sigma$. In this case, systems with longer lifetimes are more suitable for applying prognostics. For example, if a bridge failure can be predicted $3$ years in advance with an accuracy of $\pm 1$ year, that can still be a useful prediction.

However, while many uncontrolled systems can be classified as systems with long lifetimes, there exists a number of fundamental practical difficulties in performing effective prognostics for them. One of the primary issues stems directly from the typically long (often decades) lifetimes. In order to establish trust in the degradation models, they need to be adequately tested using long-term observations from “normal use” degradation or observations from properly formulated accelerated testing. With useful (from the statistical point of view) “normal use” degradation data sets being rare for many long-life system types (?), accelerated degradation efforts are common. If an accelerated degradation regime is proposed, however, what needs to be clearly demonstrated is that:

The regime can be used as a substitute for real-life degradation processes. For instance, while ? (?) and ? (?) use thermal and electrical overstress to quickly degrade electrical capacitors and predict the time of their failure (by using an empirical equation), their work does not extend to making a connection to real-life degradation, which takes place at lower temperatures and voltage/current levels. Similar issues are highlighted by ? (?) for composite materials, where mechanical, thermal, and chemical processes result in complex interactions during aging. 2. 2.

There is a known mapping from the accelerated timeline to the unaccelerated timeline. ? (?) note in an overview of condition monitoring and prognostics of insulated gate bipolar transistors that while numerous fatigue models have been constructed that predict cycles-to-failure under repetitive cycle loading, they are not designed to predict RUL under realistic usage conditions. Although use of various fatigue analysis models, e.g., ? (?), has been proposed for estimating RUL on the basis of stress cycles, their accuracy has proven difficult to confirm.

There are subfields of prognostics where accelerated aging regimes may be viable, such as in aircraft metallic structures or rotating machinery (where mechanical degradation factors could be assumed dominant). However, the issue of high uncertainty of degradation trajectories still arises, even under the same test conditions (?; ?). Finite element modeling may help alleviate degradation trajectory uncertainty in specific cases, albeit at a significant computational cost (?).

Some of the other challenges with effective prognostics for uncontrolled systems include the accuracy of estimating the actual state of health, the effects of fault interactions, and the effects of system maintenance actions (?).

If these challenges are successfully overcome and the failure mechanisms of a component are understood well enough to develop useful degradation models, a different question then arises: should the design or usage of the component be changed to mitigate these mechanisms? While in some cases this may not be feasible, in others it may be the simplest and most reliable way of improving safety and maintenance requirements (?). A redesign or change in usage would, on the other hand, make the degradation models obsolete. The next tier of degradation modes would then need to be analyzed and modeled, possibly followed by another redesign. Thus analysis intended for the development of degradation (prognostics) models instead becomes part of the design cycle.

For those uncontrolled systems that are, in fact, suitable for health management based on prognostics, the action space is typically limited to: (a) no action, (b) replacement, or (c) repair to a nominal operating condition. Even so, we still propose that predictive analysis for these systems needs to be driven by decision making requirements. For instance, if domain knowledge informs that variability in system behavior beyond some health index value $h_{\text{min}}$ is too great for choosing actions with sufficiently high confidence, then EOL can be redefined as $h_{\text{min}}$ and system dynamics beyond $h_{\text{min}}$ need to be neither modeled nor computed during prediction, potentially freeing computing resources for estimating system behavior up to $h_{\text{min}}$ with more accuracy.

4.2 Controlled systems

As soon as dynamic control is introduced into the system and uncertainty is taken into account, prognostics, as defined above, and the PHM version of the process in Figure 2 are no longer meaningful–for two key reasons. First, not having the knowledge, at $t_{p}$, of the future system actions, a PHM algorithm will either (a) have to rely on some precomputed plan to obtain $a_{t_{p}+1:H}$ for its predictive analysis (a plan that can quickly become obsolete due to outcome uncertainty) or (b) have to use a random policy (which can, for instance, result in less optimal actions being considered as probable as the more optimal ones). A random policy is also likely to result in a greater state uncertainty throughout the $[t_{p},t_{p}+H]$ interval. Second, the oft-proposed strategy of rerunning prognostic analysis after an action, so that new information can be taken into account (?), may not help. Once a suboptimal execution branch has been committed to, it may remain suboptimal regardless of future decisions. The following example provides an illustration of these issues:

Example 2.

A rover needs to traverse an area with no sunlight, going around a large crater from waypoint $wp_{0}$ to the closest suitable recharge location at $wp_{4}$. The battery charge at $wp_{0}$ is $1100\,\mathrm{Wh}$.

There are three possible levels of terrain difficulty: difficult (requiring $600\,\mathrm{Wh}$ per drive segment), moderate ($300\,\mathrm{Wh}$ per segment), and easy ($200\,\mathrm{Wh}$ per segment). All drive segments are the same length. Probabilities of terrain types in different regions are shown in Figure 4.

The rover can go to the left, $wp_{0}\rightarrow wp_{1}\rightarrow wp_{4}$, or to the right, $wp_{0}\rightarrow wp_{2}\rightarrow wp_{4}$ (left and right are relative to the diagram). If going to the right, it can decide to detour around a smaller crater $wp_{2}\rightarrow wp_{3}\rightarrow wp_{4}$ (easy terrain with $p=1.0$) instead of going directly $wp_{2}\rightarrow wp_{4}$.

A decision support PHM algorithm, running a sufficiently large number of simulations, would consider two possible execution scenarios along the left route: (L1) $e_{\text{total}}=1200\,\mathrm{Wh}$, $p=0.4$ and (L2) $e_{\text{total}}=600\,\mathrm{Wh}$, $p=0.6$ ($e_{\text{total}}$ is the total energy consumed in a scenario). The expected energy consumption along the left route can then be computed as $E[e_{\text{total, L}}]=1200\,\mathrm{Wh}\cdot 0.4+600\,\mathrm{Wh}\cdot 0.6=840\,\mathrm{Wh}$.

The algorithm would then consider four possible execution scenarios along the right route (assuming uniform random choice of action at $wp_{3}$): (R1) $e_{\text{total}}=1200\,\mathrm{Wh}$, $p=0.25$; (R2) $e_{\text{total}}=600\,\mathrm{Wh}$, $p=0.25$; (R3) $e_{\text{total}}=1000\,\mathrm{Wh}$, $p=0.25$; and (R4) $e_{\text{total}}=700\,\mathrm{Wh}$, $p=0.25$. Then $E[e_{\text{total, R}}]=(1200+600+1000+700)\cdot 0.25\,\mathrm{Wh}=875\,\mathrm{Wh}$. With $E[e_{\text{total, L}}]<E[e_{\text{total, R}}]$, the PHM algorithm commits to the left path. Note that in many cases prognostics algorithms generate even less information to support action selection than what was done here (typically $p(\text{EOL})$ only).

A DM algorithm capable of sequential reasoning under uncertainty (?) would compute $E[e_{\text{total, L}}]=840\,\mathrm{Wh}$ in the same manner as the PHM algorithm, as there are no actions needing to be selected along the left route after $wp_{0}$. On the right side, however, the DM algorithm can make an informed choice at $wp_{2}$, based on observations made along $wp_{0}\rightarrow wp_{2}$. This implies having only two possible execution scenarios: (R1) if the terrain is observed as difficult, the detour through $wp_{3}$ is taken, and (R2) if the terrain is observed as moderate, the rover goes directly to $wp_{4}$. For R1, $e_{\text{total}}=1000\,\mathrm{Wh}$, $p=0.5$. For R2, $e_{\text{total}}=600\,\mathrm{Wh}$, $p=0.5$. The expected energy use is thus $E[e_{\text{total, R}}]=(1000+600)\cdot 0.5\,\mathrm{Wh}=800\,\mathrm{Wh}$. With $E[e_{\text{total, L}}]>E[e_{\text{total, R}}]$, the algorithm chooses the right path.

Now let us assume that the true terrain condition both on the left and the right sides of the crater is difficult. The left path ($wp_{0}\rightarrow wp_{1}\rightarrow wp_{4}$) will require $1200\,\mathrm{Wh}$ to traverse, therefore a rover relying on the PHM algorithm will fall $100\,\mathrm{Wh}$ short and will not reach $wp_{4}$. A rover relying on the DM algorithm will expend only $1000\,\mathrm{Wh}$ (scenario R1), arriving at $wp_{4}$ with $100\,\mathrm{Wh}$ in reserve.

It may be suggested that the issues with the PHM approach could be eliminated if access to a precomputed operational policy $\pi_{\text{DM}}$ is provided. However, even if such a policy was available, that would still be insufficient. If, at time $t$, $p(\text{EOL})$ is computed using $\pi_{\text{DM}}$, then $a_{t+1,\text{SHM}}$ is taken on the basis of $p(\text{EOL})$, $p(\text{EOL})$ could immediately become invalid unless $T(s_{t},a_{t+1,\text{SHM}},s_{t+1})=T(s_{t},a_{t+1,\text{DM}},s_{t+1})$.

5 Unifying DM and SHM

This section discusses the benefits of a unified DM/SHM approach and outlines some of the key implementation details.

5.1 Separated vs. unified DM/SHM

The next example illustrates how unification can be helpful in balancing system health needs and operational objectives:

Example 3.

The rover starts at $wp_{0}$ in Zone 1 (Figure 6) with $500\,\mathrm{Wh}$ in the battery (out of the $1500\,\mathrm{Wh}$ capacity). The solar panels can charge the battery at a rate of $250\,\mathrm{W}$. Sunlight in Zone 1 will last for another 12 hours.

Two actions are available in the initial state $s_{0}$ at $wp_{0}$ (Figure 7): skip charging ($a_{1}=+0\,\mathrm{Wh}$) and charge to full ($a_{2}=+1000\,\mathrm{Wh}$). In Figure 7, the unitless numbers are Watt-hours of energy going in or out of the battery. Time (in hours) at each state is denoted as ’[t] h’.

The rover needs to perform a 2-hour stationary science activity at $wp_{1}$ and be able to arrive at $wp_{2}$, the next recharge point. The prior probability of the activity at $wp_{1}$ needing to be redone (a 2-hour delay) is $0.5$. Science payload power consumption is $100\,\mathrm{W}$, resulting in a net-positive ($250\,\mathrm{W}-200\,\mathrm{W}=+50\,\mathrm{W}$) power flow.

The driving times from $wp_{0}$ to $wp_{1}$ and from $wp_{1}$ to $wp_{2}$ are $4$ hours, with the average drive train power consumption of $300\,\mathrm{W}$, resulting in a net-negative ($250\,\mathrm{W}-300\,\mathrm{W}=-50\,\mathrm{W}$) power flow.

If operating without sunlight, a $150\,\mathrm{W}$ heater needs to be used to keep batteries and electronics warm, thus resulting in a net-negative power flow of $-300\,\mathrm{W}-150\,\mathrm{W}=-450\,\mathrm{W}$ for driving and $-200\,\mathrm{W}-150\,\mathrm{W}=-350\,\mathrm{W}$ for stationary science activities.

According to the general SHM policy of restoring health (battery charge, in this case) to nominal, the action chosen at $wp_{0}$ is $\pi_{shm}(s_{0})=a_{2}$ and the battery is recharged to full $1500\,\mathrm{Wh}$. After the science activity at $w_{1}$ is completed, assessment is made that it needs to be repeated. The 2-hour delay means that the entirety of $wp_{1}\rightarrow wp_{2}$ segment needs to be done without sunlight, resulting in a complete battery depletion before $wp_{2}$ is reached, with a deficit of $300\,\mathrm{Wh}$.

In computing a unified policy, however, where SHM actions are considered in the context of the overall mission, all four scenarios depicted in Figure 7 would play a role. The expected amount of battery charge remaining if $a_{1}$ is chosen would be $Q_{1}=0.5\cdot 200+0.5\cdot 300=250$. For $a_{2}$: $Q_{2}=0.5\cdot 400+0.5\cdot(-300)=50$. Action $a_{1}$ (no recharge) would be chosen and, with the two-hour delay at $wp_{1}$, the rover would arrive at $wp_{2}$ with $300\,\mathrm{Wh}$ still remaining.

The example illustrates the first benefit of unifying DM and SHM: the ability to naturally take operational objectives and constraints into account when making a system health recovery decision. The next example illustrates how DM, on the other hand, can benefit from unified action spaces and access to health-related models:

Example 4.

The rover is traveling from $wp_{0}$ to $wp_{1}$ (flat terrain) when a decision is made at point A to make a detour and attempt data collection at a scientifically valuable $wp_{2}$, requiring a 6-hour climb up a steep hill (Figure 8). The 1-hour data collection activity at $wp_{2}$ must be completed before the loss of illumination there in $10$ hours.

After completing the 2-hour climb up to point $B$ ($1/3$ of the way up), it is observed that the internal temperature of one of the drive motors has risen to $T_{m}=60^{\circ}\,\mathrm{C}$ from the nominal of $20^{\circ}\,\mathrm{C}$. At $T_{m}=80^{\circ}\,\mathrm{C}$ there is a significant risk of permanent damage and failure of the motor.

If the SHM system on the rover consists of the traditional fault detection, diagnosis, and mitigation/recovery components only, it may diagnose the fault to be increased friction, mark the component (motor) as faulty, and set a constraint on terrain types to either flat or downhill. It would then invoke the recommended action for this fault from $A_{\text{SHM}}$: stop and cool down (until $T_{m}=20^{\circ}\,\mathrm{C}$).

If a prognostic component is present, it may predict that at the current temperature increase rate, the RUL of the motor is 1 hour (with 4 hours of climb remaining to reach $wp_{2}$). The same mitigation action (stop and cool down) would be initiated and the same constraint on the current to the motor (and, thus, on the incline angle) may be set. After $T_{m}$ returns to nominal (which happens to take 1 hour), control is returned to DM. Based on the new constraints and the motor marked as faulty, DM would command the rover to abort the detour, return to $A$, and resume the drive to $wp_{2}$.

If, however, DM had stop and cool down as part of its action space ($A_{\text{DM}}$) and updated the state variables for the affected motor with the newly computed heat-up and cool down rates, an operational policy could be computed that optimizes the duration of driving and cool down intervals and allows the rover to reach $wp_{2}$ in time. For instance, if the rover drives for two hours, then stops for an hour to cool down the motor, it can still reach $wp_{2}$ in $2+1+2+1+2=8$ hours. With the science activity taking 1 hour, there would still be an hour in reserve before the loss of sunlight at $wp_{2}$.

5.2 Unification approach

In proposing the unified DM/SHM approach, we rely heavily on utility theory (?). The following, in our opinion, are the key ingredients for a successful unification: (1) a state-based system modeling framework and (2) a utility (value) function capturing the operational preferences for the system. A utility function (denoted as $U$) captures, numerically, our preferences over the space of possible outcomes. For instance, we may assign a higher utility value to a rover state where a desired scientific location has been reached. Utility can be defined for system states or for state-action pairs. If, in some system state $s$, the outcome of an action $a$ (a particular state $s^{\prime}$) is not guaranteed, then the expected utility of the $(s,a)$ tuple is

[TABLE]

In a strictly deterministic system, where a plan $\{a_{0:H}\}$ up to a horizon $H$ can be provided ahead of time, the expected utility of $s$ relative to the plan is:

[TABLE]

For systems with action outcome uncertainty, the expected utility associated with executing a policy $\pi$ for $t$ steps from state $s$ can be computed recursively as

[TABLE]

where $\gamma\in[0,1]$ is the discount factor that is sometimes used to bias towards earlier rewards, particularly in infinite horizon problems ($t\rightarrow\infty$). The optimal utility for a state can then also be computed recursively using

[TABLE]

which for $t\rightarrow\infty$ becomes the Bellman equation (?). Knowing the optimal utility function, we can derive an optimal policy:

[TABLE]

In problems with state uncertainty, beliefs $b\in B$ can take the place of states in equations 4–6 (?). The Markov property is assumed, meaning that $T(s,a,s^{\prime})$ does not depend on the sequence of transitions that led to $s$ (?).

Now that the foundational concepts have been described, we will focus on those most relevant to the proposed unification: states and the reward function $R(s,a)$. States can be vector quantities (Section 2). For real-world problems, the relevant elements of the operating environment are sometimes included in the state vector, either explicitly or implicitly (?). For instance, the rover state vector would certainly need to include the rover’s $x$ and $y$ coordinates, but may also include time $t$. These three elements allow us to indirectly access other information about the environment, e.g., solar illumination, ambient temperature, communications coverage, and terrain properties.

Similarly, health-related elements can be included in the same state vector. For the rover, the battery charge would likely be in it already for operational purposes. Adding battery temperature, however, would allow for better reasoning about the state of battery health, when combined with information on ambient temperature, terrain, and recharge current. Thus, including even a few health-related elements in the state vector may have a multiplicative effect on the amount of information available. The resulting size of the state vector may also end up being smaller than the sum of separately maintained DM and SHM state vectors, as redundant elements are eliminated.

The reward function $R(s,a)$ encodes the costs and the rewards of being in a particular state or taking a particular action in a state and can also be thought of as a local utility function. For many realistic problems with multi-variable state representations, the reward function needs to combine costs or rewards pertinent to state components. Several approaches have been proposed (?), with additive decomposition being an effective option in many cases. However it is implemented, the key property of the function is that by mapping multiple variables to a single number, it allows us to compute $U(s)$ or $U(s,a)$ and translate a potentially complex DM formulation into an abstract utility maximization problem.

As an example of such mapping, consider two different rover states: $s_{1}$, where the rate of battery discharge is higher than nominal due to a parasitic load in the electrical system and $s_{2}$, where the rover is healthy, but is traversing difficult terrain, thus also leading to a higher rate of battery discharge. The utilities $U(s_{1})$ and $U(s_{2})$ may end up being approximately equal, however, given the similar impact of the two issues on future rewards. Policies computed for $s_{1}$, $s_{2}$, and their follow-on states may be similar also and, for instance, could result in more frequent recharge stops.

One notable consequence of health-related components being integrated into a common state vector is that from the computational point of view the concepts of fault and failure become somewhat superfluous. If subsets $S_{\text{fault}}\subset S$ or $S_{\text{failure}}\subset S$ are defined for the system, the framework described above will not do anything differently for them. The only essential subset of $S$ is $S_{T}$ (the terminal states). Failure states may be part of $S_{T}$ if they result in termination of system operations; however, goal (success states) are members of $S_{T}$ also. The only difference between them is in their $U(s)$ values. As long as a component fault or a failure does not lead to a transition to a terminal state, actions that maximize the expected value of that state will be selected (which, as it happens, implements the “fail operational” philosophy).

We will refer to this unified approach as health-aware decision making (HADM). The rest of the major SHM concepts are incorporated into the new approach as follows. Fault detection and diagnostics are subsumed in belief estimation and updating, although these operations are, of course, used for nominal belief states as well. Uncertainty management can now be purposefully incorporated into the decision making process by either augmenting $A$ with information gathering actions (?), evaluated in the same context as other types of actions, or by continuing to improve $U(s)$ or $U(s,a)$ estimates until a desired level of confidence in them is reached (?). For actively controlled systems, predictive simulations are simply an integral part of $U(s)$ calculation, where $T(s,a,s^{\prime})$ serves as a one-step “prognostic” function (with degradation models, if any). Whereas prognostic algorithms applied to controlled systems are limited in their predictive ability due to the lack of knowledge about future actions, here the $U(s)$ calculation process is an exploration of possible execution scenarios, thus combining $s^{\prime}$ or $b^{\prime}$ estimation with sequential action selection.

The overall HADM operational loop (assuming state and outcome uncertainty) can be seen in Figure 9. Once the initial belief $b_{0}$ is estimated at time $t_{0}$, either an offline policy is referenced or an online policy is computed to determine $a_{0}^{*}$ (best action). The action is executed by the plant, transitioning to a new (hidden) state, and generating an observation $o_{0}$. The observation is then used to update $b_{0}$ (typically through some form of Bayesian updating) and the process repeats until a terminal state is believed to be reached.

A detailed discussion of the actual algorithms that can implement the proposed approach is left for future publications, however examples of DM algorithms that can deal with state or outcome uncertainty are provided by ? (?). In systems where both state and outcome uncertainty are not a factor, a variety of traditional state space planning algorithms (?) would not need any modification to produce plans for spaces of state vectors that include health-related components.

5.3 Emergency response vs. health management

For realistic complex systems operating in the presence of state and outcome uncertainty, $S$, $B$, and $O$ are likely to be infinitely large (with $|A|\ll\infty$, although still potentially large). The problem of finding exact optimal policies in such cases is PSPACE-complete (?). Approximate solution methods typically work online, constructing $\pi$ for the current belief $b$ based on beliefs reachable from $b$ within the decision horizon (?). They also typically perform targeted sampling from $S/B$, $A$, and $O$, thus optimality guarantees can be harder to provide. We, therefore, propose the following:

• •

That system emergency response (SER) be defined as an automated or semi-automated process that is invoked to maximize the likelihood of preserving the system’s integrity, regardless of the effect on operational goals (i.e., a different objective from HADM).

• •

That emergency response policy $\pi_{\text{SER}}$ be computed separately from the HADM policy.

In the space domain, an example of SER is putting a spacecraft in a safe mode until the emergency is resolved (?). In aviation, it could be executing recommendations of a collision avoidance system (?). Introduction of a separate SER system would likely require the introduction of $S_{\text{SER}}$, an additional $S$ subset which defines for which states $\pi_{\text{SER}}$ is invoked versus $\pi_{\text{HADM}}$. Once again, however, $S_{\text{SER}}$ will not necessarily only contain system fault and failure states. For instance, states where environmental conditions warrant emergency response (e.g., solar activity interrupting teleoperation of the rover) would be included as well. Since the scope of the SER problem is likely to be much narrower than that of HADM, it opens up the possibility of computing, verifying, and validating $\pi_{\text{SER}}$ offline (?).

As sensors, computing capabilities, and DM algorithms improve further, the fraction of the system’s state space that is under the purview of SER will decrease. Still, we foresee the need for a “safety net” SER to be there for safety-critical functions and thus advocate SER independence. SER would cover the situations where the primary HADM system may not be able to produce a suitable solution in a desired amount of time, serving as an equivalent of human reflexive responses triggered by important stimuli versus the more deliberative cognitive functionality of the brain. It could also provide dissimilar redundancy for critical regions of $S$, essentially implementing the Swiss Cheese Model (?), which advocates multiple different layers of protection for important functions.

5.4 Remarks on computational complexity

A counter-argument to the unified approach can be made that it increases the computational complexity by operating on larger $S/B$, $A$, and $O$ spaces. While this is indeed a concern, algorithms have been developed in the recent years that can effectively accommodate much of the complexity, even for problems with state and outcome uncertainty, by approximating optimal policies online (?; ?). Secondly, some of the complexity is removed from HADM into an independent SER component, making the former problem easier. Finally, as noted above, unifying the SHM and DM elements may also result in reduction of overlapping model variables.

6 Conclusions

This paper reexamines the prevalent approach to performing system health management and makes the case for why keeping system health management functions separate from decision making functions can be inefficient and/or ineffective. We also present the case for why prognostics is only meaningful in a limited set of circumstances and, even then, needs to be driven by decision making requirements.

We then explain the rationale for unifying (not just integrating) system health management with decision making and outline an approach for accomplishing that. We also propose keeping emergency response functionality separate to guarantee timely response, provide dissimilar redundancy, and allow for offline computation, validation, and verification of emergency response policies.

We believe that the proposed unification approach will improve the performance of both decision making and system health management, while potentially simplifying informational architectures, reducing sensor suite sizes, and combining modeling efforts. The approach is scalable with respect to system complexity and the types of uncertainties present. It also puts a variety of existing and emerging computational methods at the disposal of system designers.