Minimizing a stochastic convex function subject to stochastic constraints and some applications

TL;DR

This paper presents a general solution for stochastic convex function minimization with stochastic constraints, extending to sigma-finite measures and providing various applications.

Contribution

It introduces a comprehensive framework for stochastic convex optimization with constraints, accommodating dependencies and general measure spaces.

Findings

Solution to stochastic convex minimization with constraints

Extension to sigma-finite measures and dependent processes

Applications across different stochastic process scenarios

Abstract

In the simplest case, we obtain a general solution to a problem of minimizing an integral of a nondecreasing right continuous stochastic process from zero to some nonnegative random variable tau, under the constraints that for some nonnegative random variable T, tau is between zero and T a.s. and the expected value of tau is some alpha. The nondecreasing process and T are allowed to be dependent. In fact a more general setup involving sigma-finite measures, rather than just probability measures is considered and some consequences for families of stochastic processes are given as special cases. Various applications are provided.

Equations

$$\eta (\lambda )=\inf \{t|\xi (t)\ge \lambda \},$$

$${\tau }_{\lambda }=\eta (\lambda )\wedge T.$$

$$\phi (t)\ge \phi ({\tau }_{\lambda })+\lambda (t-{\tau }_{\lambda }).$$

$$\phi (t)\ge \phi ({\tau }_{\lambda +})+\lambda (t-{\tau }_{\lambda +}),$$

$$\phi ({\tau }_{\lambda })-\lambda {\tau }_{\lambda }\le \phi (t)-\lambda t$$

$$\nu {\int }_0^T\xi (s{)}^{-}ds<\infty .$$

$$\displaystyle\min\$$

$$\tau \in \mathcal{X}$$

$$\tau \in [0,T]\mu \text{-a.s.}$$

$$\mu \tau =\alpha .$$

$$q\equiv \frac{\alpha -\mu {\tau }_{\lambda }}{\mu {\tau }_{\lambda +}-\mu {\tau }_{\lambda }}.$$

$$\{{\tau }_{\lambda }\le t\}=\{\xi (t)\ge \lambda \}\cup \{T\le t\},$$

$$\mu {\tau }_{\lambda }<\alpha <\mu {\tau }_{\lambda +}.$$

$$\mu {\int }_0^{(1-q){\tau }_{\lambda }+q{\tau }_{\lambda +}}\xi (s)ds$$

$$\le (1-q)\left(\mu {\int }_0^{\tau }\xi (s)ds-\lambda (\alpha -\mu {\tau }_{\lambda })\right)$$

$$+q\left(\mu {\int }_0^{\tau }\xi (s)ds-\lambda (\alpha -\mu {\tau }_{\lambda +})\right)$$

$$=\mu {\int }_0^{\tau }\xi (s)ds.$$

$$\mu ((1-u)\tau (\alpha )+u\tau (\beta ))=(1-u)\alpha +u\beta$$

$$\nu {\int }_0^{\tau ((1-u)\alpha +u\beta )}\xi (s)ds$$

$$\le (1-u)\nu {\int }_0^{\tau (\alpha )}\xi (s)ds+u\nu {\int }_0^{\tau (\beta )}\xi (s)ds.$$

$$\sum _{i}E{\int }_0^{T_i}{\xi }_i(s{)}^{-}ds<\infty$$

$$\displaystyle\min\$$

$$\displaystyle s.t.\$$

$${\tau }_i\in [0,T_i]P\text{-a.s.},\forall i$$

$$\sum _{i}E{A}_i{\tau }_i=\alpha$$

$$q=\frac{\alpha -{\sum }_{i}E{A}_i{\tau }_{i,\lambda }}{{\sum }_{i}E{A}_i({\tau }_{i,\lambda +}-{\tau }_{i,\lambda })}.$$

$${\int }_0^{\infty }E{\int }_0^{T_s}\xi (s,t{)}^{-}dtds<\infty .$$

$$\displaystyle\min\$$

$${\tau }_s\text{ is a measurable process}$$

$$P(0\le {\tau }_s\le T_s)=1\text{for }m\text{-almost all }s\in [0,\infty )$$

$${\int }_0^{\infty }E{A}_s{\tau }_{s}ds=\alpha .$$

$$q=\frac{\alpha -{\int }_0^{\infty }E{A}_s{\tau }_{s,\lambda }ds}{{\int }_0^{\infty }E{A}_s({\tau }_{s,\lambda +}-{\tau }_{s,\lambda })ds}$$

$$m\otimes \mu {\int }_0^{(1-I){\tau }_{\lambda }+I{\tau }_{\lambda +}}\xi (s)$$

$$\le \mu {\int }_0^{\tau }\xi (s)ds=m\otimes \mu {\int }_0^{\tau }\xi (s)ds$$

$$F_e(t)=\frac{1}{ET}{\int }_0^t(1-F(s))ds$$

$${\tau }_p=F_e^{-1}(p)\wedge T$$

$$\displaystyle\min\$$

Full text

Keywords: Stochastic constrained minimization. Minimizing a stochastic convex function. Quadratic function with random coefficients. Clearing process. Constrained portfolio optimization. Neyman-Pearson lemma.

AMS Subject Classification (MSC2010): 90C15, 60G99.

1 Introduction

Motivated by the example described in Section 4, this work presents an approach to solve a certain kind of stochastic programming problems. For general reviews about stochastic programming problems see, e.g., [26, 25]. Given a general probability space, we were initially motivated by finding an optimal random variable $\tau$ that minimizes $E\int_{0}^{\tau}\xi(s)ds$, where $\xi(\cdot)$ is a nondecreasing right continuous stochastic process such that $\xi(t)<\infty$ for every $t\geq 0$, subject to two types of constraints. The first is $P(0\leq\tau\leq T)=1$ where $T$ is some random variable (possibly infinite) which is not necessarily independent of $\xi(\cdot)$. The second constraint is $E\tau=\alpha$ where $\alpha\in[0,ET]$. It turns out that there is a precise and quite clean representation of the optimal $\tau$ in terms of the pseudo-inverse process associated with $\xi(\cdot)$. In particular, $\varphi(t):=\int_{0}^{t}\xi(s)ds$ may be non-differentiable with positive probability. This makes the current work related to non-differentiable convex optimization. For references regarding deterministic non-differentiable optimization see, e.g., [6, 22] and Section 11 of [32]. For works about stochastic non-differentiable convex optimization see, e.g., [1, 3, 24]. Section 2 includes the main results of this paper. In fact, they are shown for a somewhat more general setup involving $\sigma$-finite measures rather than just probability measures. As to be shown later, this description is useful in various applications to which the other sections are devoted. Section 3 is about the case where $\xi(\cdot)$ is a deterministic function. Section 4 describes the initial motivation for the current research. It is about a Lévy-driven storage queue with a controller who picks an output rate in order to minimize the long-run average cost given a certain cost structure. Section 5 refers to a special case when $\xi(\cdot)$ is a strictly increasing linear function with random coefficients. Section 6 is about the special case where $\xi(\cdot)$ does not depend on $t$. We argue that the results for this case can be applied in hypothesis testing. Section 7 applies the main results in order to solve a problem resulting from the martingale method for solving a dynamic portfolio optimization problem in a continuous time complete market (for details see, e.g., Section 3 of [19]). Section 8 is an application of the current method to find an optimal clearing time for a general clearing model with fixed setup cost and nondecreasing holding cost function (not necessarily linear). For some background on clearing processes see (among others) [15, 16, 17, 29, 30, 31]. In Section 9 we consider the case where $\xi(\cdot)$ is a renewal counting process and $T$ is independent of $\xi(\cdot)$. When $T$ has an exponential distribution there is a particularly explicit formula for the solution. Section 10 shows that the current results can be applied to a deterministic setup of separable convex objective function with linear constraints. Finally, Section 11 is about price-regulation of an $M/G/1$ queue with customers having nonincreasing stochastic marginal utilities.

2 The main results

Denote $x^{+}=x\vee 0$, $x^{-}=-x\wedge 0$, where $x\vee y=\max(x,y)$, $x\wedge y=\min(x,y)$. Also, for some function $f$, whenever the limits exist, we denote $f(t+)=\lim_{s\downarrow t}f(s)$ and $f(t-)=\lim_{s\uparrow t}f(s)$. As usual, $\mu$-a.s. abbreviates almost surely with respect to some (sigma finite or probability) measure $\mu$ and for a sigma finite measure space $(X,\mathcal{X},\mu)$ and $\mathcal{X}$-measurable $\zeta:X\to[-\infty,\infty]$ we denote $\mu\zeta=\int_{X}\zeta d\mu$ (whenever either $\mu\zeta^{+}<\infty$ or $\mu\zeta^{-}<\infty$). For the special case where $\mu$ is a probability measure then we write $E\zeta$ (expected value) instead of $\mu\zeta$. Finally, $\nu\ll\mu$ is for $\nu$* is absolutely continuous with respect to $\mu$*.

From here on when we write $\inf\{t|t\in A\}$ we mean $\inf\{t|t\in A\cap[0,\infty)\}$. When $A\cap(0,\infty)$ is empty, the infimum is defined to be $\infty$. The following is the main idea that leads to our main result.

Lemma 1

Let $\varphi:[0,\infty)\to\mathbb{R}$ be convex, right continuous at zero (hence, continuous on $[0,\infty)$), with right derivative $\xi$ (necessarily nondecreasing and right continuous). For $\lambda\in\mathbb{R}$, denote

[TABLE]

where $\eta(\lambda)=\infty$ if $\{t|\xi(t)\geq\lambda\}$ is empty. For a given $T\in[0,\infty]$ (possibly infinite) and $\lambda\in\mathbb{R}$ let

[TABLE]

If $\tau_{\lambda}<\infty$, then for every finite $t\in[0,T]$

[TABLE]

Moreover, if $\tau_{u}<\infty$ for some $u>\lambda$ then also

[TABLE]

where $\tau_{\lambda+}=\eta(\lambda+)\wedge T$.

Observe that if we would replace $\lambda(t-\tau_{\lambda})$ on the right side of (3) by $\xi(\tau_{\lambda})(t-\tau_{\lambda})$ then the resulting inequality would be an immediate consequence of convexity (since $\xi(\tau_{\lambda})$ is a subgradient at $\tau_{\lambda}$) and would be valid for any choice of $\tau_{\lambda}$, not necessarily the one defined in (2). However, for what follows, it is important to have $\lambda$ rather than $\xi(\tau_{\lambda})$ as the multiplier.

Proof: It is well known that $\xi(t)\geq\lambda$ if and only if $t\geq\eta(\lambda)$. Therefore, $\varphi(t)-\lambda t=\int_{0}^{t}(\xi(s)-\lambda)ds$ is decreasing on $[0,\eta(\lambda))$ (empty when $\eta(\lambda)=0$) and nondecreasing on $[\eta(\lambda),\infty)$ (empty when $\eta(\lambda)=\infty$). Hence, it is decreasing on $[0,\tau_{\lambda})$ and nondecreasing on $[\tau_{\lambda},T]\cap[\tau_{\lambda},\infty)$. This implies that when $\tau_{\lambda}<\infty$, it minimizes $\varphi(t)-\lambda t$ on $[0,T]$. Thus, for every finite $t\in[0,T]$ and every $\lambda$ such that $\tau_{\lambda}<\infty$ we have that

[TABLE]

which is equivalent to (3). Clearly, (4) follows from the continuity of $\varphi$.

We now abuse the notation and instead of a function $\xi$, a nonnegative constant $T$ and a constant $\tau_{\lambda}$, from here on, these would now become functions of the form $\xi(t)=\xi(x,t)$, where we suppress the (functional) dependence on $x$. Although our main concern is with probability spaces (in which case $\xi$ is a stochastic process), it will prove useful to state the following more general result from which everything else follows. This is the main result of this paper. We will abbreviate $\zeta\in\mathcal{X}$ to mean that $\zeta$ is $\mathcal{X}$-measurable and $\tau\in[0,T]$ $\mu$-a.s. to mean that $\tau(x)\in[0,T(x)]$ for $\mu$-almost all $x\in X$.

Theorem 1

Given a measurable space $(X,\mathcal{X})$ and sigma finite measures $\nu,\mu$ such that $\nu\ll\mu$, assume that $\xi(t)\in\mathcal{X}$ for each $t\geq 0$, $\xi(t)=\xi(x,t)$ is right continuous and nondecreasing in $t$ for each $x\in X$ and $\mu$-a.s. finite for each $t>0$. Let $T=T(x)\in\mathcal{X}$ be $\mu$-a.s. nonnegative (possibly infinite) satisfying

[TABLE]

For $\alpha\in(0,\mu T)$ and $\tau=\tau(x)$, consider

[TABLE]

Let $Y=d\nu/d\mu$ be a nonnegative finite version of the Radon-Nikodym derivative. With $\tau_{\lambda}=\inf\{t|Y\xi(t)\geq\lambda\}\wedge T$, if there exists a $\lambda$ satisfying $\mu\tau_{\lambda}=\alpha$ or $\mu\tau_{\lambda+}=\alpha$, then, respectively, $\tau_{\lambda}$ or $\tau_{\lambda+}$ solves (1). Otherwise, either $\mu\tau_{\lambda}=\infty$ for all $\lambda\in\mathbb{R}$ or there exists a $\lambda$ for which $\mu\tau_{\lambda}<\alpha<\mu\tau_{\lambda+}$. If $\mu\tau_{\lambda+}<\infty$, let

[TABLE]

Then, $(1-q)\tau_{\lambda}+q\tau_{\lambda+}$ solves (1).

Observe that any version of $Y=\frac{d\nu}{d\mu}$ is $\mu$-a.s. nonnegative and finite. Thus, we can always replace it by $Y1_{\left(0,\infty\right)}(Y)$ to obtain the nonnegative finite version assumed in Theorem 1. We note that if one prefers that $\xi(t)$ is right continuous and nondecreasing $\mu$-a.s. rather than for every $x\in X$, then in addition one needs to assume in Theorem 1 that $(X,\mathcal{X},\mu)$ is complete. This is a technical nuisance which we prefer to avoid here.

Proof: We first observe that the assumption $\nu\int_{0}^{T}\xi(s)^{-}ds<\infty$ is needed in order to insure that $\nu\int_{0}^{\tau}\xi(s)ds$ is well defined (possibly infinite) for each $\mathcal{X}$-measurable $\tau\in[0,T]$ $\mu$-a.s.

Next we note that since $\xi(t)$ is nondecreasing and right continuous for each $x\in X$ and is $\mu$-a.s. finite for each $t>0$, then so is $Y\xi(t)$. Thus there is no loss of generality in assuming that $\nu=\mu$ (with $Y=1$).

We recall that from right continuity it follows that as a function of $(x,t)$, $\xi\in\mathcal{X}\otimes\mathcal{B}[0,\infty]$ (jointly measurable, e.g., Remark 1.4 on p. 5 of [13]). Here $\mathcal{B}$ is for Borel. Thus $\int_{0}^{t}\xi(s)ds\in\mathcal{X}$ for each $t\geq 0$. Since

[TABLE]

this implies that $\tau_{\lambda}\in\mathcal{X}$ for each $\lambda$ (and, in fact, that it is jointly measurable as a function of $x,\lambda$, but this will not be needed here).

If for some $\lambda$ either $\mu\tau_{\lambda}=\alpha$ or $\mu\tau_{\lambda+}=\alpha$ then we simply apply one of the inequalities (3),(4) with $t=\tau$ and integrate with respect to $\mu$, observing that $\mu\left[\lambda(\tau-\tau_{\lambda}\right]=\lambda(\alpha-\alpha)=0$ or $\mu\left[\lambda(\tau-\tau_{\lambda+})\right]=\lambda(\alpha-\alpha)=0$ (since it is required that $\mu\tau=\alpha$).

Recall that $\xi(t)$ is $\mu$-a.s. finite for every $t>0$. From (9) it follows that $\mu$-a.s. $\lim_{\lambda\to-\infty}\tau_{\lambda}=0$ and $\lim_{\lambda\to\infty}\tau_{\lambda}=T$. Thus, from monotone convergence $\mu\tau_{\lambda}\to\mu T$ as $\lambda\to\infty$ and, when $\mu\tau_{\lambda}<\infty$ for some $\lambda\in\mathbb{R}$, it converges by dominated convergence to [math] as $\lambda\to-\infty$. Thus, when $\mu\tau_{\lambda}$ is not infinite for all $\lambda$, for each $\alpha\in(0,\mu T)$ such that there is no $\lambda$ for which $\mu\tau_{\lambda}=\alpha$ or $\mu\tau_{\lambda+}=\alpha$, we can take $\lambda$ such that

[TABLE]

Assuming that the right side is finite (equivalent to $\mu\tau_{u}<\infty$ for some $u>\lambda$), then clearly, $(1-q)\tau_{\lambda}+q\tau_{\lambda+}\in[0,T]$ $\mu$-a.s. and $\mu\left[(1-q)\tau_{\lambda}+q\tau_{\lambda+}\right]=\alpha$. From (3), (4) and the convexity of $\int_{0}^{t}\xi(s)ds$ in $t$, we have that

[TABLE]

for every $\tau$ satisfying the constraints, so the proof is complete.

We observe that when $\alpha=0$, every $\mathcal{X}$-measurable $\tau\in[0,T]$ $\mu$-a.s. with $\mu\tau=0$ necessarily satisfies that $\tau=0$ $\mu$-a.s. Similarly, when $\alpha=\mu T<\infty$, every $\mathcal{X}$-measurable $\tau\in[0,T]$ $\mu$-a.s. with $\mu\tau=\mu T$ necessarily satisfies that $\tau=T$ $\mu$-a.s. Also note that for $\alpha\not\in[0,ET]$ the problem is infeasible. Thus, these cases are trivial.

We note that when $\xi(t,x)$ is strictly increasing in $t$ for every $x$, then $\tau_{\lambda}$ is continuous in $\lambda$. Hence, if $\mu\tau_{\lambda}<\infty$ for all $\lambda$ (e.g., when $\mu T<\infty$), then $\mu\tau_{\lambda}$ is continuous in $\lambda$ and for each $\alpha\in(0,\mu T)$ there is a $\lambda\in\mathbb{R}$ for which $\mu\tau_{\lambda}=\alpha$. Therefore, in this case there is no need to take a convex combination of $\tau_{\lambda}$ and $\tau_{\lambda+}$.

Proposition 1

When, in addition to the assumptions of Theorem 1, $\mu\tau_{\lambda}<\infty$ for all $\lambda\in\mathbb{R}$ (e.g. when $\mu T<\infty$), with $\tau(\alpha)$ denoting the optimum of (1) (clearly nondecreasing in $\alpha$), $f(\alpha)=\nu\int_{0}^{\tau(\alpha)}\xi(s)ds$ is a convex function of $\alpha$ on $[0,\mu T]\cap[0,\infty)$. Moreover, $\lim_{\alpha\uparrow\mu T}f(\alpha)=\nu\int_{0}^{T}\xi(s)ds$ (including the case that $T$ is not $\mu$-a.s. finite) and if $\nu\int_{0}^{\tau_{\lambda}}\xi(s)^{+}ds<\infty$ for some $\lambda$ then $\lim_{\alpha\downarrow 0}f(\alpha)=0$.

Proof: If we take $u\in(0,1)$ and some finite $\alpha,\beta\in[0,ET]$, then

[TABLE]

so that by minimality of $\tau((1-u)\alpha+u\beta)$ and convexity of $\int_{0}^{t}\xi(s)ds$ in $t$ we have that

[TABLE]

Now, we recall (see the proof of Theorem 1) that $\lim_{\lambda\to-\infty}\tau_{\lambda}=0$ and $\lim_{\lambda\to\infty}\tau_{\lambda}=T$. This implies both $\int_{0}^{\tau_{\lambda}}\xi(s)ds$ converges to zero as $\lambda\to-\infty$ and to $\int_{0}^{T}\xi(s)ds$ as $\lambda\to\infty$. Dominated (for $\lambda\to-\infty$) and monotone (for $\lambda\to\infty$) convergence (separately for $\xi(s)^{+}$ and $\xi(s)^{-}$) implies that this also holds for the integral with respect to $\nu$. Recall that we assume that $\nu\int_{0}^{T}\xi(s)^{-}ds<\infty$. Also note that since $\tau(\alpha)$ is nondecreasing in $\alpha$, then $\nu\int_{0}^{\tau(\alpha)}\xi(s)^{\pm}ds$ are nondecreasing in $\alpha$. Thus they have a limit as $\alpha$ converges to zero or to $\mu T$ (which for the latter, with $\xi(s)^{+}$, could be infinite). Thus if we take $\alpha(\lambda)=\mu\tau_{\lambda}$ then the same limits are obtained when $\lambda\to\pm\infty$.

We observe that if instead the constraint $\tau\in[0,T]$ a.s. we take $\tau\in[S,T]$ a.s. where $S\in\mathcal{X}$ satisfies $\mu S<\infty$ and $0\leq S\leq T$ $\mu$-a.s., then upon taking $\tilde{\xi}(t)=\xi(S+t)$, $\tilde{T}=T-S$ and $\tilde{\alpha}=\alpha-\mu S$, we are back to the original setup. Therefore, Theorem 1 gives a solution for this case as well. Note that for this optimization problem we may take $\xi(\cdot)$ to be indexed by $\mathbb{R}$ on and there is no need to assume that $S,T$ are nonnegative.

Finally we also observe that if $\mu$ and $\nu$ are equivalent measures, then the problem (1) may be replaced by a problem in which $\mu=\nu$, but the equality $\mu\tau=\alpha$ is replaced by $\mu A\tau=\alpha$ where $A$ is strictly positive (and finite). This implies the following two corollaries for two special cases. The first is when $\mu$ is replaced by a product measure associated with a counting measure and a probability measure and the second is where $\mu$ is replaced by the product of Lebesgue measure and a probability measure. It will be useful to refer to those in the examples that will appear later. The straightforward proofs are omitted.

Corollary 1

Given a probability space $(\Omega,\mathcal{F},P)$, assume that $\{\xi_{i}(t)|t\geq 0\}$, are nondecreasing right continuous stochastic processes with $P(|\xi_{i}(t)|<\infty)=1$ for all $t>0$ and $i$ in some finite or countable index set. Let $T_{i}$ be nonnegative (possibly infinite) random variables satisfying

[TABLE]

and let $A_{i}$ be positive and finite random variables. Consider the following stochastic optimization problem for $\alpha\in\left(0,\sum_{i}ET_{i}\right)$.

[TABLE]

Denote $\tau_{i,\lambda}=\inf\{t|\xi_{i}(t)\geq A_{i}\lambda\}\wedge T_{i}$. If there exists a $\lambda$ satisfying $\sum_{i}EA_{i}\tau_{i,\lambda}=\alpha$ or $\sum_{i}EA_{i}\tau_{i,\lambda+}=\alpha$, then, respectively, $\tau_{i,\lambda}$ or $\tau_{i,\lambda+}$ for all $i$, solve (1). Otherwise, either $\sum_{i}EA_{i}\tau_{i,\lambda}=\infty$ for all $\lambda\in\mathbb{R}$ or there exists a $\lambda$ for which $\sum_{i}EA_{i}\tau_{i,\lambda}<\alpha<\sum_{i}EA_{i}\tau_{i,\lambda+}$. If $\sum_{i}EA_{i}\tau_{i,\lambda+}<\infty$, let

[TABLE]

Then, $(1-q)\tau_{i,\lambda}+q\tau_{i,\lambda+}$, for $1\leq i\leq n$, solves (1).

Corollary 2

Given a probability space $(\Omega,\mathcal{F},P)$ and denoting the Lebesgue measure by $m$ and $ds=m(ds)$, assume that $\xi(s,t)=\xi(\omega,s,t)$ is a measurable process as a function of $(\omega,s)$ for each fixed $t\geq 0$ and nondecreasing right continuous in $t$ for each fixed $(\omega,s)\in\Omega\times[0,\infty)$, with $P(|\xi(s,t)|<\infty)=1$ for all $t>0$ and $m$-almost each $s\in[0,\infty)$. Let $T_{s}=T_{s}(\omega)\in\mathcal{B}[0,\infty)\otimes\mathcal{F}$ with $P(T_{s}\geq 0)=1$ for $m$-almost all $s\in[0,\infty)$ and assume that

[TABLE]

Finally let $A_{s}=A_{s}(\omega)$ be a measurable process and $P(0<A_{s}<\infty)=1$ for $m$-almost all $s\in[0,\infty)$. Consider the following optimization problem.

[TABLE]

Let $\tau_{s,\lambda}=\inf\{t|\xi(s,t)\geq A_{s}\lambda\}\wedge T_{s}$. If there exists $\lambda\in\mathbb{R}$ such that either $\int_{0}^{\infty}EA_{s}\tau_{s,\lambda}\,ds=\alpha$ or $\int_{0}^{\infty}EA_{s}\tau_{s,\lambda+}\,ds=\alpha$ then, respectively, $\tau_{s,\lambda}$ or $\tau_{s,\lambda+}$, for $s\in[0,\infty)$, solve (2). Otherwise, either $\int_{0}^{\infty}EA_{s}\tau_{s,\lambda}\,ds=\infty$ for all $\lambda$ or there exists some $\lambda$ for which $\int_{0}^{\infty}EA_{s}\tau_{s,\lambda}\,ds<\alpha<\int_{0}^{\infty}EA_{s}\tau_{s,\lambda+}\,ds$. When $\int_{0}^{\infty}EA_{s}\tau_{s,\lambda+}\,ds<\infty$, denote

[TABLE]

and then $(1-q)\tau_{s,\lambda}+q\tau_{s,\lambda+}$, for $s\in[0,\infty)$, solve (2).

Remark 1

It should be observed that upon taking $\tau=(1-q)\tau_{\lambda}+q\tau_{\lambda+}$ in (2) it follows that the first inequality in (2) is actually an equality. Therefore we can take a probability space $([0,1],\mathcal{B}([0,1]),m)$ (where $m$ is Lebesgue measure) and consider the random variable $I=1_{[0,q]}(\omega)$. Then take $(1-I)\tau_{\lambda}+I\tau_{\lambda+}$ on the space $([0,1]\otimes X,\mathcal{B}([0,1])\otimes\mathcal{X},m\otimes\mu)$ and obtain that

[TABLE]

for every $\tau$ satisfying the constraints. In this case we may refer to $(1-I)\tau_{\lambda}+I\tau_{\lambda+}$ as a mixed strategy. **

Remark 2

It is easy to check that if $\xi$ is indexed by $\mathbb{Z}_{+}$ (instead of $[0,\infty)$) and $\tau$ and $T$ are integer valued, then the results of this section continue to hold without change with the exception that $\tau_{\lambda}$ is defined to be $\inf\{n|\xi(n)\geq\lambda\}\wedge T$ (integer valued) and that instead of $(1-q)\tau_{\lambda}+q\tau_{\lambda+}$ (which is not necessarily an integer) we need to take a mixed strategy $(1-I)\tau_{\lambda}+I\tau_{\lambda+}$ as appearing in Remark 1. Similarly, the same is true for $\tau_{i,\lambda}$ and $\tau_{s,\lambda}$. **

Remark 3

Observe that if we replace $\mu\tau=\alpha$ in (1) by $\mu\tau\leq\alpha$, then from Lemma 1 it follows that, when finite, $\tau_{0}$ and $\tau_{0+}$ (as defined in Theorem 1), minimize $\int_{0}^{t}Y\xi(s)ds$ on $[0,T]\cap[0,\infty)$ for each $x$ and therefore it minimizes the integral with respect to $\mu$. Thus, if $\mu A\tau_{0}\leq\alpha$ then $\tau_{0}$ is an optimal solution for this modified problem. Otherwise, the optimal solution is the one given in Theorem 1. The reason is that $\lambda$ for which $E\tau_{\lambda}\leq\alpha\leq E\tau_{\lambda+}$ is necessarily negative and thus replacing $\alpha$ by $\mu\tau\in[0,\alpha]$ in (2) gives on the right side $\mu\int_{0}^{\tau}\xi(s)ds-\lambda(\mu\tau-\alpha)$. Since $\lambda<0$ it follows that $-\lambda(\mu\tau-\alpha)\leq 0$. This can also be deduced from the convexity reported in Proposition 1. Naturally, the same is valid for Corollaries 1 and 2.**

We now proceed to some examples.

3 Minimizing a deterministic convex function

When $\xi$ is deterministic we can conclude the following.

Corollary 3

Assume that $\psi:[0,\infty)\to\mathbb{R}$ is strictly convex, right continuous at zero (deterministic) and $T$ is a nonnegative, finite mean random variable with distribution $F$. Denote

[TABLE]

(stationary remaining lifetime distribution). Then for every $p\in(0,1)$

[TABLE]

minimizes

[TABLE]

Proof: Since $\psi$ is strictly convex, then its right derivative $\xi$ is strictly increasing and thus $\eta$ is continuous. Thus, there exists $\lambda$ for which $E\eta(\lambda)\wedge T=pET$ and, by Theorem 1 the optimal solution is $\eta(\lambda)\wedge T$. Now, since $\eta(\lambda)$ is a deterministic constant, then

[TABLE]

from which it follows that $\eta(\lambda)=F_{e}^{-1}(p)$, where we note that the inverse is well defined since $F_{e}$ is strictly increasing and continuous on

[TABLE]

We note that when $\psi$ is convex but not strictly convex and for some bounded below strictly convex function $\varphi$ on $[0,\infty)$ we have that $E\varphi(T)<\infty$, then for any $\tau$ satisfying $\tau\in[0,T]$ a.s. and $E\tau=pET$ we have that $E\varphi(\tau)\leq\varphi(0)\vee E\varphi(T)<\infty$ and (since $\psi+\epsilon\varphi$ is strictly convex)

[TABLE]

Upon letting $\epsilon\downarrow 0$ it follows that with the added condition that $E\varphi(T)<\infty$ for some strictly convex function on $[0,\infty)$, Corollary 3 is valid for any convex function $\psi$.

One immediate special case is minimizing $\text{Var}(\tau)$ subject to the constraints $\tau\in[0,T]$ almost surely and $E\tau=pET$ for $p\in(0,1)$. We also note that when $T$ is constant then it is easy to check that $F_{e}^{-1}(p)=pT$ and thus $\tau_{p}=pT$, as expected. This, of course, also follows from Jensen’s inequality as $\psi(\tau_{p})=\psi(E\tau)\leq E\psi(\tau)$ for any $\tau$ with $E\tau=pET$. In contrast, we recall that for this case it is well known that the maximum is given by $IT$ where $I\sim\text{Bern}(p)$. To refresh one’s memory, this follows from

[TABLE]

and then taking expected values, noting that $E\tau/T=p$.

Finally, it is interesting to note that the optimal solution in this section does not depend on the choice of the convex function $\psi$. This is not necessarily so when $\psi$ is stochastic. However, recalling that, for any convex $\psi$, an optimal solution to $\min\sum_{i=1}^{n}\psi(x_{i})$ subject to the constraints $x_{i}\geq 0$ and $\sum_{i=1}^{n}x_{i}=\alpha$ is $x_{i}=\alpha/n$ (which also does not depend on $\psi$), then perhaps we should not be too surprised.

4 Output rate control in a Lévy driven storage system

Consider a regenerative storage process with a nondecreasing Lev́y input (subordinator) such that every cycle may be split into two periods. In the first (off) the output is shut off and the workload accumulates. Consider the following cost structure. A constant holding cost per one unit of workload per one unit of time, a constant setup cost for every cycle, a constant output capacity cost rate. For further explanations regarding these costs see [10]. A controller who observes the workload level at the beginning of every on period wants to pick an output rate (which may be different from cycle to cycle) in order to minimize the long-run average cost. As explained in [10] this leads the the following optimization problem

[TABLE]

where $K_{1},K_{2},K_{3},\mu,\rho,h$ are determined by the parameters of the model and $V$ is the workload level at the beginning of an on period under the assumption $EV^{2}<\infty$. A two-phase method is applied to solve (26). In Phase I an additional constraint $EX=\alpha$ is imposed, leading to

[TABLE]

This is a special case of the setup in Section 2. For further details see [10].

5 Optimizing a quadratic function with random coefficients

Keeping with the same guideline of the example of Section 4, consider the problem

[TABLE]

for any a.s. finite random variables $A,B,C,D,T$ with $ET<\infty$, $EB^{-}<\infty$, $EC^{-}<\infty$ and $P(A>0)=P(D>0)=1$ (having an arbitrary joint distribution). Then, the assumptions of Corollary 1 are met with $n=1$, $\xi(t)=2At+B$ for every $t\geq 0$ and

[TABLE]

Thus, for this case, if $ET<\infty$ then, by continuity (and dominated convergence), for every $\alpha\in(0,EDT)$ there always is a (finite) $\lambda$ such that $ED\tau_{\lambda}=\alpha$. We also recall that for $\alpha=0$ the solution is a.s. zero and for $\alpha=ET$ it is a.s. $T$.

We also note that the special case of a uniform finite probability space results in the (deterministic) quadratic program and its solution reported in [7].

6 Optimizing a linear function with random coefficients

When the goal is to solve the following problem

[TABLE]

where $P(A>0)=P(C>0)=1$ and $EB^{-}<\infty$, we simply take $\xi(t)=A$ for every $t\geq 0$. For this case we have that $\xi(t)=A$ and thus

[TABLE]

As in the quadratic case, $A,B,C,T$ may have an arbitrary joint distribution.

Note that the same holds in the discrete time case, where we recall Remark 2. In particular, if we take $T=1$ this results in an alternative (but, admittedly, somewhat less elegant) proof of the Neyman-Pearson Lemma or, more generally, uniformly most powerful tests for this setup, where we would like to test the hypotheses

[TABLE]

where $P_{0},P_{1}$ are absolutely continuous with respect to a common $\sigma$-finite measure, under either of the constrains $P_{0}\tau=\alpha$ or $P_{0}\tau\leq\alpha$. See [2] for an in-depth treatise of such (and more general) problems which exploits convexity. In particular, compare equations (1.8)-(1.12) there to what appears here.

7 Relation to portfolio selection

In this section we refer to the classical model of dynamic utility maximization with consumption in a complete continuous-time stock market which is presented in Section 3 of [19] (see also Section 3 of [14]). In general, the model is about an agent who must dynamically decide how to manage a trade-off between consumption over time and terminal wealth. The model has a fixed finite horizon $T\in(0,\infty)$ and the agent’s initial wealth is given by a fixed parameter of the model $x_{0}\in(0,\infty)$. Furthermore, the agent’s utility from consumption of $c\geq 0$ units at time $t\geq 0$ is given by $U_{1}(t,c)$. In addition, the agent’s utility from a terminal wealth of $x\geq 0$ is $U_{2}(x)$. In addition, $U_{1}$ and $U_{2}$ are deterministic functions such that for every $c\geq 0$, $U_{1}(\cdot,c)$ is continuous and for every $t\in[0,T]$, $U_{1}(t,\cdot)$ and $U_{2}(\cdot)$ are assumed to be continuously differentiable strictly concave functions with derivatives that satisfy some additional conditions. The exact model description including the stochastic modelling of prices appears in the above-mentioned references. In these references it is shown that one approach to solving the dynamic problem is to first solve the following dual problem (in the literature it is often referred as a static problem e.g. in [19] see the last paragraph of page 37 or the first paragraph of Subsection 3.4)

[TABLE]

where $T,x_{0}>0$ are constants, $\{H_{t};t\in[0,T]\}$ is a certain $P$-a.s. positive martingale with respect to a certain filtration $\mathbb{F}$ which represents the information flow to the agent. The distribution of this martingale is determined endogenously by the model setup and defined at the beginning of Section 2.3 of [19]. Importantly, this process is not influenced by the decision variables of the optimization. In addition, we also consider the case where either $U_{1}$ or $U_{2}$ is identically zero. With these assumptions, as mentioned by [12], even when $U_{1}$ is identically zero, Lagrange multipliers for this problem may not exist and hence the Lagrange method is not always applicable. Another solution which is based on the exact definition of the process $\{H_{t};t\geq 0\}$ is provided in Section 3.4 of [19]. The solution of (33) can be obtained by our Corollary 2. In particular, one does not need to assume any differentiability assumptions on the utility functions, as is usually assumed in this literature. Moreover, since $U_{1}$ and $U_{2}$ are deterministic, then it can be seen that the solution of (33) which is specified by Corollary 2 is adapted to the filtration generated by $H(\cdot)$ and hence also adapted with respect to $\mathbb{F}$.

Furthermore, the assumption that the utility functions are deterministic can also be relaxed. Namely, take $U_{2}(\omega,x)$ to be $P$-a.s. concave in $x$ for each $\omega$. In such a case the agent has random utility from terminal wealth. For further details about models with random utilities see, e.g., [4]. For example, one can think about models with agents whose preferences are determined by a random variable which denotes the agents’ types. Now, the assumption is that for every type the utility is convex w.r.t the terminal wealth. The same can be done with respect to $U_{1}$. Now, let $S$ and $V$ be two nonnegative random variables. In addition assume that $\{\Gamma_{t};t\in[0,T]\}$ and $\{\Upsilon_{t};t\in[0,T]\}$ are two nonnegative stochastic processes. In particular, assume that these random quantities are exogenous to the model, i.e. they are not influenced by the choice of $X$ and $\{c_{t};t\geq 0\}$. The requirement that $X$ and $\{c_{t};t\geq 0\}$ are nonnegative could be replaced by the constraint

[TABLE]

to which the results of Section 2 still apply. Examples of models considering such constraints are, e.g., [11, 18, 20, 21]. Another case which is also covered by the current work is when $T=\infty$ and $U_{2}$ is identically zero. Finally, note that this kind of optimizations is also motivated by discrete time models (see, e.g., Section 3.3 of [5]).

8 Optimal clearing times in a regenerative clearing process

As described by [29], “a stochastic clearing system is characterized by a non-decreasing stochastic input process , where $Y(t)$ is the cumulative quantity entering the system in $[0,t]$, and an output mechanism that intermittently and instantaneously clears the system, that is, removes all the quantity currently present.” In particular, the clearing system is regenerative if the workload which is associated with this system is a regenerative process. Such systems have been extensively studied in the literature, e.g., see [9, 16, 28].

Now, if $\xi$ is a nonnegative process then we can think of $\{\xi(t)|0\leq t<\tau\}$ as the first cycle of a (regenerative) clearing process. When $E\tau<\infty$, for such a clearing process an ergodic distribution exists and if $\xi^{*}$ has this distribution then we have that for any nonnegative Borel $g$,

[TABLE]

Note that if $g$ is a nonnegative, nondecreasing and right continuous function then $g(\xi(\cdot))$ is a nonnegative, nondecreasing right continuous process and we can apply the results of Section 2 to optimize $Eg(\xi^{*})$ subject to the constraints in (1). This also provides a method for solving the following optimization problem for any given $K>0$ and nonnegative, nondecreasing right continuous $g$:

[TABLE]

In this case the cost structure is a setup cost $K$ incurred right after each clearing and a (possibly nonlinear) holding cost function $g$. The solution is obtained by first restricting the minimization to feasible $\tau$’s satisfying $E\tau=\alpha\in(0,ET]$ to obtain

[TABLE]

where $\tau(\alpha)$ denotes the optimal solution from Theorem 1 and then $h$ is minimized over $(0,ET]$ either analytically, when possible, or numerically.

Finally we observe that for every $a,b\geq 0$ such that $a+b>0$ and every $x,y$ we have that

[TABLE]

With the notations from (8), setting

[TABLE]

we infer that

[TABLE]

This implies (recall (36)) that

[TABLE]

Therefore, for this optimization problem it suffices to restrict attention to random times of the form $\tau_{\lambda}$, for $\lambda$ such that $E\tau_{\lambda}>0$. Note that $E\tau_{\lambda}>0$ if and only if $P(\tau_{\lambda}>0)>0$. Since $\eta(\lambda)>0$ if and only if $\lambda>g(\xi(0))$, we can replace ‘$\lambda|E\tau_{\lambda}>0$’ on the right hand side of (8) by ‘$\lambda|P(g(\xi(0))<\lambda,T>0)>0$’. In particular, when $\xi(0)$ independent of $T$ (in the literature it is usually assume to be zero, so that this independence is automatic), then this results in ‘$\lambda|\lambda>g(\xi(0))$’.

9 Example: Renewal counting process $\xi$ with independent $T\sim\exp(\mu)$ and a bit more

Assume that $T\sim\exp(\theta)$ is independent of $\{\xi(t);t\geq 0\}$ which is a renewal counting process with inter-renewal times distributed like some $X$. As usual, it is assumed that $P(X\geq 0)=1$ and $P(X=0)<1$. Since $\xi(0)=0$, then $\eta(\lambda)=0$ for every $\lambda\leq 0$ and, for every $\lambda\geq 0$,

[TABLE]

where $S_{\lceil\lambda\rceil}$ is the $\lceil\lambda\rceil$th renewal time. Therefore, for every $\lambda\geq 0$, $\tau_{\lambda}=T\wedge S_{\lceil\lambda\rceil}$, so that

[TABLE]

For integer valued $\lambda$ we have that $\tau_{\lambda+}=\tau_{\lambda+1}$ and otherwise $\tau_{\lambda+}=\tau_{\lambda}$. It is easily verified that with

[TABLE]

for $\alpha\in(0,\theta^{-1})$, we either have that $E\tau_{\lambda_{\alpha}}=\alpha$ or $E\tau_{\lambda_{\alpha}+}=E\tau_{\lambda_{\alpha}+1}=\alpha$ or $E\tau_{\lambda_{\alpha}}<\alpha<E\tau_{\lambda_{\alpha}+1}$, in which case the optimal solution is $(1-q)\tau_{\lambda_{\alpha}}+q\tau_{\lambda_{\alpha}+1}$ where $q$ is given by (8).

If $T$ has a finite mean and is independent of $\xi$ but does not have an exponential distribution, then

[TABLE]

where $f_{e}(t)=(1-F_{T}(t))/ET$ and $F_{T}$ and $F_{X}$ are the cumulative distribution functions of $T$ and $X$, respectively. In this case there is no explicit formula for $\lambda_{\alpha}$, but in many cases it can be computed numerically. A case which is worth pointing out is when $X\sim\exp(\theta)$. In this case it can be easily verified that

[TABLE]

where $T_{e}$ has a distribution with density $f_{e}$. In this case

[TABLE]

and the $k$th derivative of this function with respect to $\theta$ is given by

[TABLE]

Thus, in principal, the knowledge of $Ee^{-sT}$ for every $s\geq 0$ gives us a procedure for finding everything that is needed in order to compute the optimal $\tau$ in this case.

10 Separable convex optimization with linear constraints

Obviously, Corollary 1 can be applied to the following optimization problem in which $f_{i}$ are convex (not necessarily differentiable) functions, $t_{i}$ nonnegative reals (possibly infinite) and $a_{i}$ are strictly positive and finite.

[TABLE]

When $f_{i}$ are differentiable, $t_{i}=\infty$, $a_{i}=1$ and $\alpha=1$, the results are consistent with the famous Gibb’s Lemma, noting that if $\xi_{i}(0)\geq\lambda$ then necessarily $\eta_{i}(\lambda)=0$. This is a standard convex optimization problem with a separable objective function and linear constraints and the number of references is huge (for the case where $f_{i}$ are differentiable). For example, quite a few examples are given in [23]. The standard solution (under differentiability assumptions) is by applying the Karush-Kuhn-Tucker conditions or Gibb’s Lemma.

11 Regulation of $M/G/1$ system

Consider a single-server first-come-first-served $M/G/1$ queue with an arrival rate $\lambda$ where the service demand distribution is determined endogenously by the following mechanism: Each customer decides during service when to terminate that service and leave the system. This decision is influenced by three factors. Linear waiting time cost (excluding service time), a marginal utility modelled by a general nonincreasing right-continuous stochastic process which is observed from the moment service begins until departure from the system and a price the customer pays the system which is some function of the time this customer occupies the server. Now, customers must join the queue, so reneging or abandonments are not allowed. Therefore, their decisions concern only the question of when to quit service after it has begun. Hence, for every customer the decision is an optimal stopping problem with respect to the information generated by their marginal utilities. Since the marginal utilities of the customers are iid processes which are independent from the arrival process, then so are the resulting decisions of the customers. Thus, eventually the model under examination is a regular $M/G/1$ queue with service distribution which is determined endogenously by the mechanism above. The question is how to determine a price function which implies optimal resource allocation from a social point of view? Assume that the social optimality criterion is the expected utility of a customer in steady-state for the resulting $M/G/1$ system for certain family of price functions. Now, instead of solving this problem directly, [8] first optimizes the performance measure over all service distributions. Then, once an optimal service distribution is derived, it turns out to be possible to construct a price function for which the corresponding optimal stopping times of the customers are distributed according to the optimal service distribution which was initially derived. If $V(\cdot)$ is a process with the distribution of the marginal utilities of the customers and we assume that $V(0)$ is a nonnegative random variable such that $EV^{2}(0)<\infty$, then an optimal service distribution is a solution of the problem

[TABLE]

To solve this problem, as in Sections 4 and 8, [8] includes a two-phase method. Given some $\alpha\in\left[0,\lambda^{-1}\right)$, Phase I solves (49) with an additional constraint $ES=\alpha$, i.e.,

[TABLE]

This is a special case of the optimization solved in Section 2.