Optimal Impulse Control of a Simple Reparable System in a Nonreflexive Banach Space

TL;DR

This paper develops a mathematical framework for optimal impulse control in a reparable system modeled by complex equations in a nonreflexive Banach space, focusing on minimizing failure probability through state-dependent controls.

Contribution

It introduces a novel approach to impulse control in a nonreflexive Banach space setting, proving existence and deriving optimality conditions for the control problem.

Findings

Existence of an optimal impulse controller is rigorously proven.

First-order necessary conditions for optimality are derived using variational inequalities.

Control inputs are designed to depend on system state to ensure nonnegativity.

Abstract

We discuss the problem of optimal impulse control representing the preventive maintenance of a simple reparable system. The system model is governed by coupled transport and integro-differential equations in a nonreflexive Banach space. The objective of this paper is to construct nonnegative impulse control inputs at given system running times that minimize the probability of the system in failure mode. To guarantee the nonnegativity of the controlled system, we consider the control inputs to depend on the system state. This essentially leads to a bilinear control problem. We first present a rigorous proof of existence of an optimal controller and then apply the variational inequality to derive the first-order necessary conditions of optimality.

Equations

$$\frac{d{p}_0(t)}{dt}=-{\lambda }_1{p}_0(t)+{\int }_0^l{\mu }_1(x)p_1(x,t)dx,$$

$$\frac{\partial p_1(x,t)}{\partial t}+\frac{\partial p_1(x,t)}{\partial x}=-{\mu }_1(x)p_1(x,t),$$

$$p_1(0,t)={\lambda }_1{p}_0(t),$$

$$p_0(0)=1,p_1(x,0)=0.$$

$${\hat{p}}_1(t)={\int }_0^l{p}_1(x,t)dx,t\ge 0.$$

$${\int }_0^{l^{\prime }}{\mu }_1(x)dx<\infty ,l^{\prime }<l,\text{and}{\int }_0^l{\mu }_1(x)dx=\infty .$$

$$\frac{d{p}_0(t)}{dt}=-{\lambda }_1{p}_0(t)+{\int }_0^l{\mu }_1(x)p_1(x,t)dx+\sum _{i=1}^N\delta (t-t_i){\int }_0^l{u}_i(x)dx,$$

$$\frac{\partial p_1(x,t)}{\partial t}+\frac{\partial p_1(x,t)}{\partial x}=-{\mu }_1(x)p_1(x,t)-\sum _{i=1}^N\delta (t-t_i)u_i(x),$$

$$(P,Q)=p_0{q}_0+{\int }_0^l{p}_1{q}_{1}dx,$$

$$u_i(x)=b_i(x)p_1(x,t_i),x\in [0,l],i=1,2\dots ,N,$$

$$J(b_1,b_2,\dots ,b_N)=$$

$$+\frac{1}{2}|{\int }_0^l{p}_1(x,T)dx{|}^2,$$

$$\displaystyle\mathcal{A}P=\left[\begin{array}[]{c}-\lambda_{1}p_{0}+\int_{0}^{l}\mu_{1}(x)p_{1}(x)\,dx\\ -(\frac{d}{dx}+\mu_{1}(x))p_{1}(x)\end{array}\right],$$

$$\displaystyle D(\mathcal{A})=\big{\{}P\in X\big{|}\ \frac{dp_{1}(x)}{dx}\in L^{1}[0,l],\int^{l}_{0}\mu_{1}(x)p_{1}(x)\,dx<\infty,\ \text{and}\ p_{1}(0)=\lambda_{1}p_{0}\big{\}}.$$

$$\dot{P}(t)=\mathcal{A}P(t)+\sum _{i=0}^{N}f(t-t_i),t>0,$$

$$P(0)=(p_0(0),p_1(x,0){)}^T\ge 0,$$

$$p_0+{\int }_0^l{p}_1(x,0)dx=1.$$

$$P(t)=$$

$$=$$

$$+\mathcal{S}(t-t_2){\left[{\int }_0^l{b}_2(x)p_1(x,t_2)dx,-b_2(x)p_1(x,t_2)\right]}^T$$

$$+\cdots +\mathcal{S}(t-t_N){\left[{\int }_0^l{b}_N(x)p_1(x,t_N)dx,-b_N(x)p_N(x,t_N))\right]}^T.$$

$$P(t)=\mathcal{S}(t-t_i)P(t_i)+\mathcal{S}(t-t_i){\left[{\int }_0^l{b}_i(x)p_1(x,t_i)dx,-b_i(x)p_1(x,t_i)\right]}^T,$$

$$P(t)=\mathcal{S}(t)P(0),$$

$$P(t)=\mathcal{S}(t-t_N)P(t_N).$$

$$\frac{d{p}_0(t)}{dt}+\frac{d}{dt}{\int }_0^l{p}_1(x,t)dx=0.$$

$$p_0(t)+{\int }_0^l{p}_1(x,t)dx=p_0(0)+{\int }_0^l{p}_1(x,0)dx=1,\forall t\ge 0,$$

$$\Vert P(t){\Vert }_X=\Vert P(0){\Vert }_X=1,\forall t\ge 0.$$

$$P(t)$$

$$=\mathcal{S}(t-t_i){\left[1-{\int }_0^l(1-b_i(x))p_1(x,t_i)dx,(1-b_i(x))p_1(x,t_i)\right]}^T,$$

$$P(t_i^{+})-P(t_i)=$$

$$+\mathcal{S}(t_i^{+}-t_i){\left[{\int }_0^l{b}_i(x)p_1(x,t_i)dx,-b_{i1}(x)p_1(x,t_i)\right]}^T-P(t_i)$$

$$=$$

$$\displaystyle U_{ad}=\{U=[b_{1},b_{2},\dots,b_{N}]^{T}\in(L^{2}[0,l])^{N}\big{|}$$

$$\text{is equicontinuous on}[0,l]\}.$$

$$(\dot{P}(t),Q)=(\mathcal{A}P(t),Q)+(\sum _{i=0}^{N}f(t-t_i),Q),\forall Q=[q_0,q_1{]}^T\in X^{\ast },$$

Full text

1 Introduction

Reparable systems occur naturally in many real world problems such as product design, inventory systems, computer networking and complex manufacturing processes, etc. In recent years, mathematical models governed by distributed parameter systems of coupled partial and ordinary hybrid equations have been widely used to study the reliability of reparable systems (cf. [3, 6, 8, 9, 10, 11, 12, 23]). Reliability is defined as the probability that the system, subsystem or component will operate successfully by a given time $t$ (cf. [2, 20]). This paper is mainly concerned with the optimal control problem of a reparable multi-state system introduced by Chung [3], which is described as coupled transport and integro-differential equations. In particular, we consider that the system has one failure state in our current work. This will be sufficient to capture the essence of the control design. The precise model of system equations is described by

[TABLE]

with the boundary condition

[TABLE]

and initial conditions

[TABLE]

Here $p_{0}(t)$ stands for the probability that the device is in good state, represented as [math], at time $t$; $p_{1}(x,t)$ stands for the probability density (with respect to the repair time) that the failed device is in failure state, represented by $1$, and has an elapsed repair time of $x$ at time $t$, where $x\in[0,l]$ with $l<\infty$; $\lambda_{1}$ stands for the constant failure rate of the system for failure mode $1$; $\mu_{1}(x)$ stands for the time-dependent repair rate when the device is in state $1$ and has an elapsed repair time of $x$; probability $\hat{p}_{1}(t)$ that the failed device is in state $1$ at time $t$ is defined by

[TABLE]

The following assumptions are associated with the device:

The failure rates are constant; 2. 2.

All failures are statistically independent; 3. 3.

All repair times of failed devices are arbitrarily distributed; 4. 4.

The repair process begins soon after the device is in failure state; 5. 5.

The repaired device is as good as new; 6. 6.

No further failure can occur when the device has been down.

Furthermore, we assume that the repair rate has the following properties

[TABLE]

The pointwise and steady-state availability of the uncontrolled multi-state system was discussed in [3] by solving the inversion of the Laplace transformation. However, this approach used two potential assumptions that the system has a nonnegative time-dependent solution and the solution is asymptotically stable, which are nontrivial when the repair rate is time dependent. Xu, Yu and Zhu in [23] provided a rigorous mathematical framework for proving the well-posedness and asymptotic stability of the system by using $C_{0}$-semigroup theory. It is proved that the system operator generates a positive $C_{0}$-semigroup of contraction, therefore the system has a unique nonnegative time-dependent solution. It is also shown that 0 is a simple eigenvalue of the system operator and the unique spectral point on the imaginary axis. In particular, the system is conservative in the sense that the sum of the probabilities of the system in good mode and failure mode is always $1$. Moreover, Hu, Xu, Yu and Zhu in [12] showed that the $C_{0}$-semigroup is quasi-compact and irreducible. As a result, it follows that the time-dependent solution converges to the steady-state solution exponentially, which is the positive eigenfuction corresponding to the simple eigenvalue [math]. However, the previous work is based on the assumption that the repair time $l=\infty$, which is not realistic. Neither the system can be under repair nor the server can work forever. Recently, Hu in [13] considered $l<\infty$ and further proved that the $C_{0}$-semigroup is eventually compact and eventually differentiable. In this case, condition (6) indicates that if the system, subsystem or component can not be repaired over the finite repair time interval $[0,l]$, then it will be replaced by the new one immediately. Furthermore, the problems of controllability and enhancement of stabilizability of this system by distributed controls have been discussed in [14, 21].

This paper aims at minimizing the probability of the system at failure mode by employing a maintenance policy, which is interpreted as the control inputs. Maintenance is defined as any action that restores failed units to an operational condition or retains non-failed units in an operational state (cf. [15, 16]). These actions affect the overall performance of the system such as reliability, availability, downtime, cost of operation, etc. A proper maintenance policy and a feasible approach are crucial. The optimal control design formulated in this paper provides insight into the development of such a policy. In general, there are three types of maintenance actions: corrective maintenance, preventive maintenance and inspections. Corrective maintenance serves to restore a failed system to operational status. As we can see from the model equations (1)–(2), repair rate $\mu_{1}(x)$ plays the role of corrective maintenance. Since a component’s failure time is not known a priori, $\mu_{1}(x)$ is performed during unpredictable intervals. This usually involves replacing or repairing the component that is responsible for the failure of the overall system.

Compared to corrective maintenance, the concept of preventive maintenance is to replace components or subsystems before they fail in order to promote continuous system operation. Cost needs to be taken into account in preventive maintenance since financially it is more sensible to replace parts or components that have not failed at predetermined intervals rather than to wait for a system failure. The latter may result in a costly disruption in operations. In our current work, we propose a preventive maintenance policy that are applied at the given system running times. To formulate a meaningful strategy, two criteria need to be satisfied in our control design:

The controlled system is nonnegative and conservative; 2. 2.

The control inputs are nonnegative.

The paper is organized as follows. In section 2, we introduce an impulse control design to the reparable system and establish the well-posedness of the controlled system in a nonreflexive Banach space. Then we prove the existence of an optimal control in section 3. In section 4, we present the first-order necessary conditions of optimality by using a variational inequality.

2 Optimal Impulse Control Design

Consider that the reparable system (1)–(4) is controlled by the impulse control inputs at given system running times $t_{i},0\leq t_{1}<t_{2}<\dots<t_{N}\leq T$, for some $T>0$. Let $u_{i}(x)\geq 0,x\in[0,l],i=1,2,\dots,N,$ be the corresponding input intensities, which represent the the update/replacement rate when the system is in failure state at time $t_{i}$ with an elapsed repair time $x$. The controlled system is described by

[TABLE]

where $\delta(t-t_{i})$ is the Dirac $\delta-$function supported at $t_{i}$. Note that the system will not be updated/replaced if it is in good state. To study the behavior of the reparable system in terms of probabilities, we consider the state space $X=\mathbb{R}\times L^{1}[0,l]$ with $\|\cdot\|_{X}=|\cdot|+\|\cdot\|_{L^{1}[0,l]}$, which is a nonreflexive Banach space. Let $X^{*}$ be the dual of $X$, then $X^{*}=\mathbb{R}\times L^{\infty}[0,l]$. The duality between $X$ and $X^{*}$ is defined by

[TABLE]

for every $P=[p_{0},p_{1}]^{T}\in X$ and $Q=[q_{0},q_{1}]^{T}\in X^{*}$. The objective of this paper is to establish an optimal update/replacement policy that minimizes the probability $\hat{p}_{1}(t)$ of the system in failure mode over $[0,T]$, where $\hat{p}_{1}(t)$ is defined by (5).

To be physically meaningful, we seek for the control inputs $u_{i}(x)\geq 0$ with $x\in[0,l]$, such that the controlled system is nonnegative and conservative. In particular, we consider that $u_{i}$ depends on the probability density of the system in failure mode at $t_{i}$, that is,

[TABLE]

where $0\leq b_{i}(x)\leq 1$, stands for the updated/replaced rate depending on the elapsed repair time $x$. In other words, the input intensity $u_{i}(x)$ at time $t_{i}$ is up to the probability density of the device in failure mode with an elapsed repair time of $x$ at time $t_{i}$. For a given final time $T>0$, the current work aims at deriving the optimal distribution of $b_{i}(x)$ such that the following cost functional is minimized:

[TABLE]

where $\beta_{i}\geq 0,i=1,2,\dots,N,$ are the control weight parameters. This essentially becomes a bilinear optimal control problem.

We first show that the controlled system is nonegative and conservative. Define the system operator $\mathcal{A}$ and its domain

[TABLE]

[TABLE]

Let $\mathcal{S}(t)$, $t\geq 0,$ denote the $C_{0}$-semigroup generated by $A$, which is a positive semigroup of contraction and eventually compact and eventually differentiable for $l<\infty$ [13, 23]. Moreover, [math] is a simple eigenvalue of $\mathcal{A}$ and the only spectrum on the imaginary axis.

Now let $f_{0}(t-t_{i})=\delta(t-t_{i})\int^{l}_{0}b_{i}(x)p_{1}(x,t_{i})\,dx$, $f_{1}(x,t-t_{i})=-\delta(t-t_{i})b_{i}(x)p_{1}(x,t_{i})$, and $f(x,t-t_{i})=(f_{0}(t-t_{i}),f_{1}(x,t-t_{i}))^{T}$. Then the controlled system (7)–(8) can be rewritten as an abstract Cauchy initial value problem in state space $X$

[TABLE]

with a general initial condition

[TABLE]

satisfying

[TABLE]

The problem of impulse control has been discussed in [1] and the references cited therein. The solution of Cauchy problem (14)–(16) can be given by the variation of parameter formula

[TABLE]

In particular, if the control input is only exerted at $t=0$, then it becomes a start control problem. We summarize the property of the solution in the following theorem.

Theorem 2.1

For $t\in(t_{i},t_{i+1}],1\leq k\leq N-1$, the function given by the variation of parameter formula

[TABLE]

is the mild solution of the Cauchy initial value problem (14). Moreover, for $t\in(0,t_{1}]$,

[TABLE]

and for $t\in(t_{N},\infty)$,

[TABLE]

Note that integrating the second equation of (7) with respect to $x$ from [math] to $l$, and then adding them to the first equation result in

[TABLE]

This implies that

[TABLE]

or

[TABLE]

Thus the system is conservative with respect to $\|\cdot\|_{X}$. In other words, the sum of probability distributions of the controlled system is always $1$ for every $t\geq 0$.

Next replacing $p_{0}(t_{i})$ by $1-\int^{l}_{0}p_{1}(x,t_{i})\,dx$ in (17), we have

[TABLE]

which is nonnegative for $0\leq b_{i}(x)\leq 1$. Further note that the solution has jumps at $t_{i},i=1,\dots,N$. In fact,

[TABLE]

Therefore, we have $P(t)\in PWC_{l}([0,T];X)$, where $PWC_{l}([0,T];X)$ denotes the space of piecewise continuous functions on $[0,T]$ with values in $X$, that are left continuous and possess righthand limits.

3 Existence of an Optimal Solution

In this section, we address the existence of an optimal control to problem (10) subject to the controlled system (14)–(16). Since the controlled system does not have any “smoothing effects” on state $p_{1}(\cdot,t_{i})$, we need an additional condition imposed on $b_{i}$ to have the compactness of the admissible control set in order to handle the bilinear term $b_{i}(\cdot)p_{1}(\cdot,t_{i})$. To this end, we define the set of admissible controls by

[TABLE]

Next we introduce the weak solution to (14)–(16).

Definition 3.1

For $P_{0}\in X$ with $\|P_{0}\|_{X}=1$, $P=[p_{0},p_{1}]^{T}\in PWC_{l}([0,T];X)$ is said to be a weak solution of system (14)–(16), if $P$ satisfies

[TABLE]

and the initial condition

[TABLE]

The following theorem provides the existence of an optimal solution.

Theorem 3.2

There exists an optimal solution $(U^{*},P^{*})$ of problem (10) subject to the controlled system (14)–(16) in the sense of Definition 3.1.

Proof. Since $J$ is bounded from below, we can choose a minimizing sequence $\{U^{k}\}\subset U_{ad}$ for each $i=1,\dots,N$ such that

[TABLE]

By the definition of $U_{ad}$, the sequence $\{U^{k}\}$ is uniformly bounded and equicontinuous in $U_{ad}$. With the help of Ascoli’s Theorem [19], we may extract a subsequence, still denoted by $\{U^{k}\}$, such that

[TABLE]

Let $P^{k}$ be the solutions of (26) corresponding to $U^{k}$ and satisfying the initial condition $P^{k}(0)=P_{0}$. Note that $P^{k}\in PWC_{l}([0,T],X)$ and $\|P^{k}(t)\|_{X}=1$ for any $t\geq 0$ based on (21). Thus for $t=t_{i}$, there exists a subsequence, still denoted by $\{P^{k}\}$, satisfying

[TABLE]

Next we show that $P^{*}$ is the solution corresponding to $U^{*}$. According to Definition 3.1,

[TABLE]

and $P^{k}(0)=P_{0}$. Note that

[TABLE]

where $A^{*}$ is the adjoint operator of $A$ defined by

[TABLE]

with its domain

[TABLE]

Let $\psi=[\psi_{0},\psi_{1}]^{T}$ be a vector of continuously differential functions on $[0,T]$ with $\psi_{j}(T)=0,j=0,1$. For each $Q\in X^{*}$, we multiply (31) by $\psi$ and integrate the left hand side of (31) by parts with respect to $t$. This process yields

[TABLE]

where by virtue of (29)–(30), we have

[TABLE]

Moreover,

[TABLE]

where we used the fact that $b^{*}_{i}(x)q_{1}(x,t_{i})\in L^{\infty}[0,l]$ for the second term of (37) converging to zero.

Now pass to the limit in each term of (31) by using (35)–(37). As a result, we have

[TABLE]

where $f^{*}(t-t_{i})=[\delta(t-t_{i})\int^{l}_{0}b^{*}_{i}(x)p^{*}_{1}(x,t_{i})\,dx,-\delta(t-t_{i})b_{i}(x)p_{1}(x,t_{i})]^{T}$. It remains to be shown that $P^{*}(0)=P_{0}$. Consider

[TABLE]

We repeat the same process as above. Multiplying (39) by a continuously differentiable function $\psi$ with $\psi(T)=0$ and integrating by parts yield

[TABLE]

Comparing (40) with (38) gives

[TABLE]

Choose $\psi$ with $\psi(0)=1$. Then (41) becomes

[TABLE]

and thus $P^{*}_{0}=P_{0}$. Lastly, by the lower semicontinuity of $J$ for all $U\in U_{ad}$, we have

[TABLE]

This completes the proof.

4 Optimality Conditions

In this section, the first-order necessary optimality conditions for problem (10) will derived by using a variational inequality [18]. If $U$ is an optimal solution of problem (10), then

[TABLE]

where $J^{\prime}(U)\cdot h$ stands for the G$\hat{a}$teaux derivative of $J$ at $U$ in the direction $h\in U_{ad}$.

Define operator $D\colon C([0,T],L^{1}[0,l])\to C[0,T]$ by

[TABLE]

According to the definition of $J$ in (10), we have

[TABLE]

for $i=1,2\dots,N$, where $z_{0i}=p^{\prime}_{0}(u_{i})\cdot h_{i}$, $z_{1i}=p^{\prime}_{1}(u_{i})\cdot h_{i}$, and $h=[h_{1},h_{2},\dots,h_{N}]^{T}\in U_{ad}$. Note that the G$\hat{a}$teaux derivatives $z_{0i}$ and $z_{1i}$ satisfy the following equations

[TABLE]

with boundary conditions

[TABLE]

and initial conditions

[TABLE]

The well-posedness of (44)–(47) can be established by using the similar approach as in Theorem 2.1.

Finally, the first-order necessary conditions of optimality are given by the following theorem.

Theorem 4.1

Assume that $U^{*}=[b^{*}_{1},b^{*}_{2},\dots,b^{*}_{N}]^{T}\in U_{ad}$ is an optimal solution of (10) subject to (14)–(16). Let $[p_{0},p_{1}]^{T}\in PWC_{l}([0,T];X)$ be the corresponding solution and $[q_{0i},q_{1i}]^{T}\in C[0,T]\times PWC_{r}([0,T];L^{\infty}[0,l]),i=1,2,\dots,N$, be the solutions to the following adjoint systems

[TABLE]

Then $b^{*}_{i}$ satisfies

[TABLE]

where $PWC_{r}([0,T];L^{\infty}[0,l])$ denotes the space of piecewise continuous functions on $[0,T]$ with values in $L^{\infty}[0,l]$, that are right continuous and possess lefthand limits.

Proof. We first compute the first term in (43). This yields

[TABLE]

Note that by (45) and (48) we have

[TABLE]

and

[TABLE]

Thus combining (52) with (53)–(54) yields

[TABLE]

Replacing $\int^{T}_{0}(D^{*}Dp_{1}(x,t),z_{1i})\,dt$ in (43) by (55) and making use of the condition (50), we have the G$\hat{a}$teaux derivative $J^{\prime}_{i}(U)\cdot h_{i}$ become

[TABLE]

for $i=1,2,\dots,N,$ and any $h=[h_{1},h_{2},\dots,h_{N}]^{T}\in U_{ad}$. Finally, combining (56) with the constraint that $0\leq b_{i}\leq 1$, we get

[TABLE]

This completes the proof.

5 Conclusion

An impulse control design is discussed for a simple reparable system in a nonreflexive Banach space, which represents a preventive maintenance policy. First-order conditions of optimality are derived for solving the optimal solution. A finite difference scheme that preserves the nonnegativity and conservativeness will be developed to discretize the controlled system and a gradient decent based algorithm will be constructed to implement the control design in our future work.

Acknowledgments

H. Xu was supported by NSAF grant No. U 1430125.