Asymptotics and Renewal Approximation in the Online Selection of Increasing Subsequence

TL;DR

This paper analyzes the online selection of increasing subsequences from a Poisson process, providing detailed asymptotic expansions and a new proof of the central limit theorem using renewal approximation.

Contribution

It introduces a renewal approximation approach and refines asymptotic estimates for the moments of the subsequence length under optimal strategies.

Findings

Asymptotic expansions for moments of subsequence length

A novel proof of the central limit theorem using renewal approximation

Enhanced understanding of the distribution of selected subsequence lengths

Abstract

We revisit the problem of maximising the expected length of increasing subsequence that can be selected from a marked Poisson process by an online strategy. Resorting to a natural size variable, the problem is represented in terms of a controlled partially deterministic Markov process with decreasing paths. Refining known estimates we obtain fairly complete asymptotic expansions for the moments, and using a renewal approximation give a novel proof of the central limit theorem for the length of selected subsequence under the optimal strategy.

Equations

$$\sqrt{2t}-\log (1+\sqrt{2t})+\tilde{c}<v(t)<\sqrt{2t}.$$

$$v(t)=\sqrt{2t}-\frac{1}{12}\log t+c^{\ast }+\frac{\sqrt{2}}{144\sqrt{t}}+O(t^{-1}),t\to \infty ,$$

$$\mathrm{Ein}(t)={\int }_0^t\frac{1-e^{-s}}{s}\mathrm{d}s\sim \log t,t\to \infty .$$

$$\sqrt{3}\frac{L_0(t)-\sqrt{2t}}{(2t{)}^{1/4}}\stackrel{\mathrm{d}}{\to }\eta ,$$

$$\mathbb{E}\left(\sum _i{\pi }_i\right)\to \max ,\mathrm{subject}\mathrm{to}\mathbb{E}\left(\sum _i{\pi }_i{\xi }_i\right)\le 1,{\pi }_i\in \{0,1\},$$

$$\mathbb{E}\left(\sum _i{\pi }_i(1-\lambda {\xi }_i)\right).$$

$$1=\mathbb{E}\left(\sum _i{\xi }_{i}1({\xi }_i<1/\lambda )\right)=\frac{t}{2{\lambda }^2}.$$

$$\psi (t,s,y)=(1-y)\phi ((t-s)(1-y))$$

$$0<\frac{x-y}{1-y}\le \phi ((t-s)(1-y)).$$

$$v^{\prime }(t)={\int }_0^1(v(t(1-x))+1-v(t){)}_{+}\mathrm{d}x,v(0)=0,$$

$$v(t(1-x))+1-v(t)=0.$$

$$u(z):=v(z^2)$$

$$u^{\prime }(z)=4{\int }_0^z(u(z-y)+1-u(z){)}_{+}(1-y/z)\mathrm{d}y,u(0)=0.$$

$$u(z-y)+1-u(z)=0.$$

$$u^{\prime }(z)=4{\int }_0^{{\theta }^{\ast }(z)}(u(z-y)+1-u(z))(1-y/z)\mathrm{d}y,u(0)=0.$$

$$\mathcal{I}g(z)=4{\int }_0^z(g(z-y)+1-g(z){)}_{+}(1-y/z)\mathrm{d}y.$$

$$(u(z_1-y)+1-u(z_1){)}_{+}<((g(z_1-y)+c)+1-(g(z_1)+c){)}_{+},$$

$$\mathcal{I}{u}_1(z)=4{\int }_0^{{\theta }_1(z)}(u_1(z-y)+1-u_1(z))(1-y/z)\mathrm{d}y\to \frac{2}{{\alpha }_1},z\to \infty .$$

$${\theta }_2(z)\sim \frac{1}{\sqrt{2}}-\frac{{\alpha }_2}{2(z+1)}.$$

$$\mathcal{I}{u}_2(z)\sim \sqrt{2}-\left(\frac{1}{3}+{\alpha }_2\right)\frac{1}{z+1}.$$

$${\alpha }_2=-\left(\frac{1}{3}+{\alpha }_2\right),$$

$$u(z)\sim \sqrt{2}z-\frac{1}{6}\log z.$$

$$u_3(z):=\sqrt{2}z-\frac{1}{6}\log (z+1)+\frac{{\alpha }_3}{z+1}.$$

$${\theta }_3(z)\sim \frac{1}{\sqrt{2}}+\frac{1}{12(z+1)}.$$

$$\mathcal{I}{u}_3(z)\sim \sqrt{2}-\frac{1}{6(z+1)}+\left({\alpha }_3-\frac{1}{36\sqrt{2}}-\frac{1}{3}\right)\frac{1}{(z+1{)}^2}.$$

$$u_3^{\prime }(z)=\sqrt{2}-\frac{1}{6(z+1)}-\frac{{\alpha }_3}{(z+1{)}^2}$$

$$u(z)=\sqrt{2}z-\frac{1}{6}\log z+O(1),z\to \infty ,$$

$$\lambda (z):=\theta (z)-\frac{{\theta }^2(z)}{2z}.$$

$$\tilde{Z}(s):=\sqrt{(1-Y(s))(t-s)},s\in [0,t],$$

$$\sigma (s):=t^{1/2}-\tilde{Z}(s)+\sum _{s^{\prime }\le s}\Delta \tilde{Z}(s^{\prime }).$$

Full text

1. Introduction

Suppose a sequence of independent random marks with given continuous distribution is observed at times of the unit-rate Poisson process. Each time a mark is observed it can be selected or rejected, with every decision becoming immediately final. What is the maximum expected length $v(t)$ of increasing subsequence, which can be selected over a given horizon $t$ in online fashion by a nonanticipating strategy?

Samuels and Steele [16] introduced this stochastic optimisation problem as offspring of its counterpart with fixed sample size and used similarity between the two problems to obtain the leading asymptotics, $v(t)\sim\sqrt{2t}$ for $t\to\infty$. Remarkably, the square-root order of growth was concluded from superadditivity, similarly to the seminal Hammersley’s argument for the longest (offline in our context) increasing subsequence [15].

The best known to date bounds are

[TABLE]

Analogous upper bounds are common in folks literature [2, 3, 4, 7, 11]. The lower bound (for $t$ not too small), with explicit constant $\tilde{c}$, appeared in Bruss and Delbaen [5]. In the same paper Bruss and Delbaen also obtained estimates on the variance of the length and in [6] they proved a functional limit theorem for this and other characteristics of the optimal selection process.

The approach in [5, 6] drew heavily on the concavity of $v(t)$ and involved martingale arguments. In this paper we revisit the problem in light of the Samuels-Steele observation [16] that changing the scale to $z=t^{1/2}$ yields an approximate linearisation. We improve upon (1) by obtaining an asymptotic expansion for the optimal expected length

[TABLE]

similarly refine known estimates of the variance and show that there is a simple selection strategy within $O(1)$ from the optimum. The finer asymptotics (2) up to a bounded term is obtained by bootstrapping the optimality equation similarly to the method used in [4]. But justifying convergence of the remainder and the expansion beyond $O(1)$ require much more probabilistic insight. Our main novelty here is a representation of the selection problem in terms of a controlled piecewise deterministic Markov process in one dimension. On this way, we will use comparison with an alternating renewal process to explain the logarithmic term in (2) and to give a new proof of the central limit theorem for the length of increasing subsequence under the optimal and some suboptimal strategies.

Notation. We will use $\sim$ for asymptotic expansions written without estimate of the remainder, e.g. $f(t)\sim f_{1}(t)+f_{2}(t)+\dots+f_{k}(t)$ as $t\to\infty$ means that $f_{i+1}(t)=o(f_{i}(t))$ for $1\leq i<k$. We denote $c^{*},c_{0},c_{1}$ etc some absolute constants, and $c$ a constant with context-dependent value.

2. Planar Poisson setup and the leading asymptotics

Standardising the distribution of marks to $[0,1]$-uniform leads to a natural setting of the problem with horizon $t$ in terms of the unit-rate Poisson random measure in the rectangle $[0,t]\times[0,1]$. The generic atom $(s,x)$ is interpreted as mark $x$ observed at time $t$, whereupon a selection/rejection decision must be made solely on the base of the allocation of atoms within $[0,s]\times[0,1]$. A sequence $(s_{1},x_{1}),\dots,(s_{n},x_{n})$ of atoms is said to be increasing if it is a chain in the partial order in two dimensions, that is $0<s_{1}<\dots<s_{n}$ and $0<x_{1}<\dots<x_{n}$. The task is to maximise the expected length of an increasing sequence over selection strategies adapted to the aforementioned information.

To solve the optimisation problem it is sufficient to consider a relatively small class of strategies defined recursively by means of some acceptance window $\psi(t,s,y)$ satisfying $0\leq\psi(t,s,y)\leq 1-y$ for $0\leq s\leq t<\infty$ and $y\in[0,1]$. The corresponding strategy selects observation $(s,x)\in[0,t]\times[0,1]$ if and only if $0<x-y\leq\psi(t,s,y)$, where $y$ is the running maximum, i.e. the last (hence the highest) mark selected before time $s$, with the convention that $y=0$ if no selections have been made. Note that the running maximum process and the selected chain uniquely determine one another.

The acceptance window can be regarded as a control function for the running maximum, which is a right-continuous Markov process $Y=(Y(s),~{}0\leq s\leq t)$ starting with $Y(0)=0$, with piecewise constant paths increasing by positive jumps. At time $s$ in state $y$ a transition occurs at rate $\psi(t,s,y)$, and if $Y$ jumps the increment $Y(s)-Y(s-)$ is uniformly distributed on $[0,\psi(t,s,y)]$. For the optimal process, the expected number of jumps is maximal.

Intuitively, a large acceptance window steers $Y$ from [math] to about $1$ in just a few jumps. On the other hand, a small acceptance window makes the jumps rare, so the time resource expires before a substantial number of selections is made.

For instance, the greedy strategy has the largest possible acceptance window $\psi(t,s,y)=1-y$. The strategy selects the sequence of records [8], which has the expected length given by the exponential integral function

[TABLE]

The greedy strategy is optimal for $t\leq 1.345\cdots$, when the expected number of records is not bigger than 1.

A stationary strategy has acceptance window of the form $\psi(t,s,y)=(1-y)\wedge\delta(t)$, depending neither on the time of observation nor on the running maximum, as long as $Y$ does not overshoot $1-\delta(t)$. The asymptotic optimality in the principal term is achieved within this class. In the remaining part of this section, we sketch a proof which absorbs some ideas from the previous work [3, 4, 11, 16].

Choose $\delta=\delta(t)$. Up to the first moment when a selected mark exceeds $1-\delta$, the running maximum $Y$ coincides with a compound Poisson process $S$, characterised by the jump rate $\delta$ and the $[0,\delta]$-uniform distribution of increments. For $\delta\to 0$ but $t\delta\to\infty$, the number of jumps of $S$ over time $t$ is asymptotic to $t\delta$ (in the mean-square sense), and the number of jumps until $S$ passes $1-\delta$ is asymptotic to $2/\delta$. The maximum of $(t\delta)\wedge(2/\delta)$ is attained for $\delta^{*}(t)=\sqrt{2/t}$, which results in the expected length asymptotic to $\sqrt{2t}$. After the first selection above $1-\delta^{*}$ the strategy is greedy, with the expected number of choices being $O(1)$, hence not affecting the leading asymptotics.

In the sequel under the stationary strategy we shall mean the one with $\delta=\delta^{*}$, that is with $\psi(t,s,y)=(1-y)\wedge\sqrt{2/t}$. This strategy maintains a balance between increasing on the marks and time scales, so that the running maximum $Y$ fluctuates about the linear function $s/t$, and both resources are exhausted almost simultaneously.

Let $L_{0}(t)$ be the length of increasing sequence chosen by the stationary strategy. Representing $L_{0}(t)$ as a minimum of two independent renewal processes we have a limit

[TABLE]

with $\eta=\xi_{1}\wedge(\sqrt{3}\xi_{2})$, where $\xi_{1}$ and $\xi_{2}$ are independent standard normal variables. Specialising formulas for moments found in [14], ${\mathbb{E}}\eta=-\sqrt{2/\pi}$, ${\rm Var}(\eta)=2-2/\pi$.

Now, for the compound Poisson process $S$ controlled by $\delta^{*}$, the expected number of jumps is exactly equal to $\sqrt{2t}$. To prove the upper bound (1) we will show that $S$ can be identified with the optimal chain in an online selection problem with a weaker mean-value constraint. To that end, consider a problem of online selection from the Poisson random measure in unbounded domain $[0,t]\times[0,\infty)$, but with the restriction that observation $(s,x)$ is available for selection only if $0<x-y\leq 1$, where $y$ is the running maximum at time $s$. Suppose the objective is to maximise the expected length of the selected chain subject to the constraint that the mean mark of the ultimate selection does not exceed $1$. Clearly, every strategy with choices from the bounded rectangle $[0,t]\times[0,1]$ is admissible also in the extended scenario; in particular, it has the last selected mark not exceeding 1 almost surely. In the extended setting, the observations satisfying the chain condition arrive by a unit-rate Poisson process independent of the marks. Whichever selection strategy, if $y$ is the last mark chosen before time $s$, the next (if any) mark which comes in question is $y+\xi$, where $\xi$ is $[0,1]$-uniformly distributed and independent of the previous observations. Let $(s_{i},\xi_{i})$ be the increasing sequence of observation times of such marks and $\xi_{i}$ their associated uniform variables. A decision on observation at time $s_{i}$ can be encoded into a 0-1 random variable $\pi_{i}$ adapted to the Poisson random measure within $[0,s_{i}]\times[0,\infty)$. With this notation the task becomes a version of the knapsack problem

[TABLE]

with the Lagrange function

[TABLE]

Here, $\pi_{i}$’s enter linearly hence the maximum is attained by the indicators $\pi_{i}^{*}=1(\xi_{i}<1/\lambda)$. The multiplier $\lambda^{*}=(t/2)^{1/2}$ is found from the constraint

[TABLE]

Since $1/\lambda^{*}=\delta^{*}$, the selected chain has the same distribution as the compound Poisson process $S$, whence the upper bound in (1).

3. The optimality equation

The stationary strategy lacks the following self-similarity feature inherent to the overall optimal strategy. If at time $s$ the running maximum is $y$, further selections are to be made from $[s,t]\times[y,1]$, which is an independent subproblem, equivalent to the original problem in $[0,t^{\prime}]\times[0,1]$ with $t^{\prime}=(t-s)(1-y)$. This implies that it is sufficient to optimise over the class of self-similar strategies with acceptance window of the form

[TABLE]

for some $\varphi:[0,\infty)\to[0,1]$. Such a strategy accepts mark $x$ at time $s$ if and only if

[TABLE]

The dynamic programming principle leads to the optimality equation for the maximal expected length $v(t)$,

[TABLE]

The optimal acceptance window is $\psi^{*}(t,s,y)=(1-y)\varphi^{*}((t-s)(1-y))$, where $\varphi^{*}(t)=1$ if $v(t)\leq 1$ (when $v(t)$ is given by (3)), and otherwise $\varphi^{*}(t)$ is defined implicitly as the unique solution to

[TABLE]

See [5] for the derivation of (6), properties and estimates of $\varphi^{*}(t)$ and $v(t)$. Our focus is on the asymptotic expansion for large $t$.

With the change of variable

[TABLE]

the optimality equation (6) becomes equation of convolution type

[TABLE]

We set $\theta^{*}(z)=z$ if $u(z)\leq 1$, and otherwise define $\theta^{*}(z)$ to be the unique solution to

[TABLE]

By monotonicity we can re-write (7) as

[TABLE]

4. Asymptotics I Let ${\cal I}g$ be the integral operator acting on functions $g\in C^{1}[0,\infty)$ as

[TABLE]

In this notation equation (7) becomes $u^{\prime}={\cal I}u$. Note that we can write ${\cal I}g(z)=4\int_{0}^{\theta(z)}(\cdots){\mathrm{d}y}$ if the equation $g(z-y)+1-g(z)=0$ has a unique solution $y=\theta(z)$.

The following lemma resembles a familiar comparison method to estimate solutions of differential equations (see [10], section 9.1).

Lemma 1**.**

If $g^{\prime}(z)>{\cal I}g(z)$ for all sufficiently large $z$ then $\limsup\limits_{z\to\infty}(u(z)-g(z))<\infty$. Likewise, if $g^{\prime}(z)<{\cal I}g(z)$ for all sufficiently large $z$ then $\liminf\limits_{z\to\infty}(u(z)-g(z))>-\infty$.

Proof . Observe that ${\cal I}g={\cal I}(g+c)$ for constant $c$. Since $u(z)$ increases to $\infty$ we may choose $z_{0}$ large enough to satisfy both $z-\theta^{*}(z)>z_{0}$ and $g^{\prime}(z)>{\cal I}g(z)$ for $z>2z_{0}$. Assume to the contrary that $\limsup\limits_{z\to\infty}(u(z)-g(z))=\infty$. Choosing $c$ large enough we will achieve that $u(z)<g(z)+c$ for $z_{0}<z<z_{1}$, and $z_{1}>2z_{0}$ for $z_{1}:=\min\{z>z_{0}:u(z)=g(z)+c\}$, which exists by the assumption. Then for $y<z_{1}$

[TABLE]

hence my monotonicity of the integral $u^{\prime}(z_{1})={\cal I}u(z_{1})\leq{\cal I}(g+c)(z_{1})={\cal I}g(z_{1})<g^{\prime}(z_{1})$. But this is a contradiction since $u^{\prime}(z_{1})\geq(g+c)^{\prime}(z_{1})=g^{\prime}(z_{1})$ by definition of $z_{1}$ as the location where $u$ first reaches $g+c$. The second part of the lemma is argued similarly. $\Box$

We will now compare the solution to (7) with various test functions. Let $u_{1}(z):=\alpha_{1}z$. We have $u^{\prime}_{1}(z)=\alpha_{1}$ and for $\theta_{1}(z):=1/\alpha_{1}$,

[TABLE]

The match $\alpha_{1}={2}/{\alpha_{1}}$ occurs at $\alpha_{1}^{*}:=\sqrt{2}$, thus by the lemma $\limsup\limits_{z\to\infty}(u(z)-u_{1}(z))<\infty$ for $\alpha_{1}>\sqrt{2}$ and therefore $\limsup\limits_{z\to\infty}u(z)/z\leq\sqrt{2}$. Likewise, the second part of the lemma yields $\liminf\limits_{z\to\infty}u(z)/z\geq\sqrt{2}$. These bounds imply $u(z)\sim\sqrt{2}\,z$.

We try next functions $u_{2}(z):=\sqrt{2}z+\alpha_{2}\log(z+1)$ (we take $\log(z+1)$ and not $\log z$ to avoid the annoying singularity at [math]). Solving $u_{2}(z-y)+1-u_{2}(z)=0$, for large $z$ we get expansion

[TABLE]

We may proceed with only the first term in (10) since the second makes a negligible $O(z^{-2})$ contribution to ${\cal I}u_{2}(z)$ which expands as

[TABLE]

With $u_{2}^{\prime}(z)=\sqrt{2}+\alpha_{2}/(z+1)$ the match occurs when

[TABLE]

that is for $\alpha_{2}^{*}:=-1/6$. It follows from the lemma that $(u(z)-\sqrt{2}z)/\log(z+1)\to\alpha_{2}^{*}$, that is

[TABLE]

To further refine the approximation we try

[TABLE]

This time we need to calculate with higher precision, hence take two terms

[TABLE]

Expanding the integrand and integrating:

[TABLE]

To match with

[TABLE]

we must choose $\alpha_{3}^{*}:=1/6+\sqrt{2}/144$. Taking $\alpha_{3}$ bigger or smaller than $\alpha_{3}^{*}$, allows us to sandwich $u$. However, our comparison method based on the lemma only yields

[TABLE]

since the third term in (11) is already bounded. A different approach will be applied to show convergence of the $O(1)$ remainder.

5. Piecewise deterministic Markov process We represent next the selection problem by means of a piecewise deterministic Markov process in one dimension.

Let $\theta:(0,\infty)\to(0,\infty)$ be a function satisfying $0<\theta(z)\leq z$, and let

[TABLE]

The following rules define a piecewise deterministic Markov process $Z$ on $[0,\infty)$ with continuous drift component and random instantaneous jumps:

• (i)

the process decreases continuously with unit speed,

• (ii)

the jumps are negative and occur at rate $4\lambda(z)$, for $z>0$,

• (iii)

if a jump from state $z$ occurs, the jump size has density $(1-y/z)/\lambda(z)$ with support $[0,\theta(z)]$,

• (iv)

the process terminates upon reaching [math].

We denote $Z|z$ this process starting in position $z$. The path of $Z|z$ can be constructed by thinning the set of arrivals of an inhomogeneous marked Poisson process with intensity (ii) and marks distributed as in (iii). The following procedure is similar to many familiar parking, packing and scheduling models in applied probability. Let $z_{1}$ be the rightmost arrival on $[0,z]$ with some mark $y_{1}$. Call $(z_{1},z]$ a drift interval comprised of drift points, and call $z_{1}$ a jump point. Remove all arrivals from the gap $(z_{1}-y_{1},z_{1}]$. Then iterate thinning arrivals of the Poisson process, to the left of $z_{1}-y_{1}$ in place of $z$. If after some iteration a jump point $z_{k}$ cannot be found, the drift interval extends from the last defined point $z_{k-1}-y_{k-1}$ (or $z$ in case $k=1$) to [math]. The union of drift intervals corresponds to the range of $Z|z$, while the gaps are skipped by jumps. The jump points of $Z|z$ divide $[0,z]$ in cycles. A cycle, in the right-to-left order, is comprised of a drift interval followed by a gap. The exception is the leftmost drift interval adjacent to [math]. In this picture, the time variable is unambiguously introduced by requiring that the time to pass $z^{\prime}\leq z$ is equal to the Lebesgue measure of the range intersected with $[z^{\prime},z]$.

To connect to the increasing subsequence problem choose horizon $t$ and let $Y$ be the running maximum process under some self-similar strategy (5). Note that $(1-Y(s))(t-s)$ is the area of the rectangle, from which selections after time $s$ can be made. Let

[TABLE]

which is a drift-jump process decreasing from $t^{1/2}$ to [math], with negative jumps $\Delta\widetilde{Z}(s)=\widetilde{Z}(s)-\widetilde{Z}(s-)$ at times of selection. The decay of $\widetilde{Z}$ due to the drift is a strictly increasing continuous process

[TABLE]

For $\sigma^{\leftarrow}$ the inverse function to $\sigma$, consider the time-changed process

[TABLE]

Identifying the drift rate and jump distribution it is seen that (14) is the process $Z|\sqrt{t}$, with $\theta$ found by matching the jump rates as

[TABLE]

In particular, $Y$ over horizon $t=z^{2}$ has the same number of jumps as $Z|z$. This reduces the optimal selection problem with horizon $t$ to choosing a control function $\theta$ with the objective to maximise the expected number of jumps of $Z|\sqrt{t}$.

Denote $N_{\theta}(z)$ the number of jumps of the process $Z|z$ steered by given function $\theta$, and let $u_{\theta}(z):={\mathbb{E}}N_{\theta}(z)$. With probability $4\lambda(z){\mathrm{d}z}$ the process moves from a small vicinity of $z$ to $z-y$, with $y$ sampled from the density in (iii), in which case the expected number of jumps is equal to $u_{\theta}(z-y)+1$. Otherwise, the process drifts through to $z-{\mathrm{d}z}$. This decomposition readily yields equation

[TABLE]

For the general $\theta$, the integrand in (15) need not be positive and even monotonicity of $u_{\theta}$ may not hold. In purely analytic terms, for any fixed $z$, maximising $u_{\theta}(z)$ over admissible $\theta:[0,z]\to(0,\infty)$ is the problem of calculus of variations. The solution is $\theta=\theta^{*}$, defined implicitly by the optimality equation (7) and (8).

We shall assume throughout that $\theta$ is bounded and differentiable. That the optimal $\theta^{*}$ is bounded can be seen at this stage of our analysis from (8) and (13).

The asymptotic comparison method based on an analogue of Lemma 1 works for (15) as well. In particular, for

[TABLE]

we obtain the same expansion as (13).

The decreasing sequence of jump points of $Z|z_{0}$ is a Markov chain with terminal state [math]. Let $U_{\theta}(z_{0},\cdot)$ be the occupation measure on $[0,z_{0}]$ counting the expected number of jump points, in particular $U_{\theta}(z,[0,z])=u_{\theta}(z)$. Denote $p(z_{0},z),$ for $0\leq z\leq z_{0}$ the probability that $z$ is a drift point, in particular $p(z_{0},z_{0})=p(z_{0},0)=1$. There is a jump point within ${\mathrm{d}z}$ only if $z$ does not belong to a gap, hence the occupation measure has a density which factorises as

[TABLE]

Lemma 2**.**

There exists a pointwise limit $p(z):=\lim\limits_{z_{0}\to\infty}p(z_{0},z)$, which satisfies

[TABLE]

with some positive constants $a$ and $\alpha$.

Proof.

The proof is by coupling. Choose constant $\overline{\theta}$ big enough to have $\sup\theta(z)<\overline{\theta}$. Fix $z<z_{0}<z_{1}$ with $z>2\overline{\theta}$ (the latter assumption does not affect the result). Consider two independent processes $Z_{0}$ and $Z_{1}$ with $Z_{0}\stackrel{{\scriptstyle d}}{{=}}Z|z_{0},~{}Z_{1}\stackrel{{\scriptstyle d}}{{=}}Z|z_{1}$. Define $Z^{\prime}$ by running the process $Z_{1}$ until it hits a drift point $\xi$ of $Z_{0}$ , then from this point on switch over to running $Z_{0}$. Such a point $\xi$ exists since both processes have a gap adjacent to [math]. By the strong Markov property, $Z^{\prime}$ has the same distribution as $Z_{1}$. If the coupling occurs at some $\xi\in[z,z_{0}]$, the point $z$ is of the same type (drift or jump) for both $Z^{\prime}$ and $Z_{0}$.

The coupling does not occur within $[z,z_{0}]$ only if $Z_{0}$ and $Z_{1}$ have no common drift points within these bounds. Given that $y>z$ is a drift point, the probability that the drift interval covering $y$ extends to the left over $y-\overline{\theta}$ is at least $\pi$, for some constant $\pi>0$. This follows since the length of drift interval dominates stochastically an exponential random variable with rate $\sup 4\lambda(z)<\infty$. In particular, the rightmost drift interval, adjacent to $z_{0}$, is shorter than $\overline{\theta}$ with probability at most $1-\pi$, in which case the rightmost cycle is shorter than $2\overline{\theta}$. Given $\xi$ is not in the first cycle, the probability that $\xi$ is not in the second is again at most $1-\pi$, in which case also the second cycle is shorter than $2\overline{\theta}$. Continuing so forth we see that $\xi\notin[z,z_{0}]$ with probability at most $(1-\pi)^{k}$ for $k=\lfloor(z_{0}-z)/(2\overline{\theta})\rfloor$. This readily implies an exponential bound $|p(z_{0},z)-p(z_{1},z)|<ae^{-\alpha(z_{0}-z)}$, uniformly in $z_{1}>z_{0}$. Sending $z_{0}\to\infty$ we see that $p(z_{0},z)$ is a Cauchy sequence, whence the claim. ∎

In the terminology of random sets, $p(z_{0},\cdot)$ is the coverage function (see [13] p. 23) for the range of $Z|z_{0}$. As $z_{0}\to\infty$ the range converges weakly to a random set ${\cal Z}\subset[0,\infty)$, comprised of infinitely many intervals separated by gaps. Indeed, let $A(z_{0},z)\leq z$ be the maximal point of the range of $Z|z_{0}$ within $[0,z]$, for $z\leq z_{0}$. The coupling argument in the lemma also shows that $A(z_{0},z)$ has a weak limit, $A(z)$, which is sufficient to justify convergence of the range intersected with $[0,z]$, due to the Markov property. By Sheffé’s lemma $U_{\theta}(z_{0},\cdot)$ converges weakly to some $U_{\theta}$, which is the occupation measure for the point process of left endpoints of intervals making up $\cal Z$.

6. Reward processes Suppose each jump point of $Z|z$ is weighted by some location-dependent reward $r$. Let $w_{\theta,r}(z)$ be the total expected reward accumulated by $Z|z$ controlled by $\theta$. By analogy with (15) we have equation

[TABLE]

On the other hand, we can write $w_{\theta,r}(z)$ as the average over the occupation measure,

[TABLE]

Lemma 3**.**

For $r$ integrable function, the solution to (16) has a finite limit

[TABLE]

If $|r(z)|=O(z^{-\beta})$ as $z\to\infty$ for some $\beta>1$ then $|w_{\theta,r}(z)-\rho_{\theta,r}|=O(z^{-\beta+1})$.

Proof.

Since $p(z_{0},z)\lambda(z)<\overline{\theta}$ the existence of limit follows from (17), (18) and Lemma 2 by the dominated convergence. The convergence rate is estimated by splitting the difference as

[TABLE]

where the second integral is of the order $O(z^{-\beta+1})$ while the first is of the lesser order $O(e^{-\alpha z/2})$ by Lemma 2. ∎

7. Asymptotics II Differentiating (7) with an account of (8) we obtain

[TABLE]

where

[TABLE]

Since $\theta^{*}(z)=z$ for small $z$ this has a simple pole at 0, but the singularity is compensated in (17), so Lemma 3 and (13) ensure that

[TABLE]

With (20) at hand, expanding in (8) we get $\theta^{*}(z)=\sqrt{1/2}+O(z^{-1})$. Replacing $\theta^{*}$ by $\sqrt{1/2}$ in (7) incurs remainder of smaller order $O(z^{-2})$ because $\theta^{*}$ is the stationary point of the integral. Recalling that $u_{2}$ (with $\alpha_{2}^{*}=-1/6$) satisfies $u_{2}^{\prime}(z)={\cal I}u(z)+O(z^{-2})$, for the difference $w=u-u_{2}$ we obtain equation (16) with $r(z)=O(z^{-2})$, hence $u(z)-u_{2}(z)$ by Lemma 3 approaches a finite limit at rate $O(z^{-1})$ as $z\to\infty$. This proves an expansion

[TABLE]

with some constant $c^{*}$.

Our methods are not geared to identify $c^{*}$, because the initial value $u(0)=0$ was nowhere used, but changing it (e.g., by resorting to a selection problem with terminal payoff) will result in adding $u(0)$ to $c^{*}$. Nevertheless, with some more effort it is possible to go beyond $O(1)$. Let us first estimate the variation of $u^{\prime}$.

Lemma 4**.**

For fixed $\overline{d}>0$, as $z\to\infty$

[TABLE]

Proof.

Using the integral representation (17) of $u^{\prime}$ with $r(z)=O(z^{-2})$, write

[TABLE]

The first integral is obviously $O(z^{-2})$ uniformly in $d\leq\overline{d}$. By Lemma 2 the second is estimated as

[TABLE]

using Laplace’s method. ∎

The lemma applied to the right-hand side of (19) gives $u^{\prime\prime}(z)=O(z^{-2})$. In (9) we replace $\theta^{*}$ by $\sqrt{1/2}$, expand $u(z-y)-u(z)=-yu^{\prime}(z)+O(z^{-2})$ and integrate to obtain with some algebra

[TABLE]

Expanding similarly in (8) we get a finer formula for the optimal control function

[TABLE]

in accord with (12). Since $u_{3}^{\prime}(z)={\cal I}u_{3}(z)+O(z^{-3})$ the difference $w=u-u_{3}$ satisfies (16) with $r(z)=O(z^{-3}))$, hence invoking Lemma 3 we obtain $u(z)-u_{3}(z)=\hat{c}+O(z^{-2})$ for some constant $\hat{c}$. This must agree with (21), therefore $\hat{c}=c^{*}$. Thus we have shown

Theorem 5**.**

For the optimal process, the control function $\theta^{*}$ satisfies (22), and the expected number of jumps has expansion

[TABLE]

To appreciate the effect of the second term in (22) it is helpful to consider control functions of the kind

[TABLE]

The parameter appears in the asymptotics of solutions to (15) as

[TABLE]

(whichever $u_{\theta}(0)$ that only affects the constant).

Constant $c$ in (25) does not exceed $c^{*}$ in (23), but the relation between the $z^{-1}$-terms can be the opposite. For instance, for $\theta_{0}(z)=\sqrt{1/2}\wedge z$ we have

[TABLE]

8. The variance For $N_{\theta}(z)$, the number of jumps of $Z|z$ driven by $\theta$, let $w(z)={\mathbb{E}}(N_{\theta}(z))^{2}$ be the second moment. This function satisfies

[TABLE]

Integrating the inhomogeneous term this can be reduced to the form (16), but with $r(z)$ of the order of $z$. This equation has properties similar to (7), that is adding a constant to $w$ also gives a solution with some other initial value, and the right-hand size increases in $w(z-y)$. Hence an analogue of Lemma 1 can be applied to compare $w$ with various test functions.

We shall consider first the case of optimal $\theta=\theta^{*}$. It is an easy exercise to see that $w(z)\sim 2z^{2}$, hence the leading term in the integrand is $-4zy+2\sqrt{2}z$, which vanishes at $y=\sqrt{1/2}$. For this reason the $O(z^{-2})$ remainder in (22) will contribute to the solution only $O(1)$, and not $O(\log z)$ as one might expect. Using this fact and (22) it is possible to match the sides of the equation by selecting coefficients of the test function

[TABLE]

achieving that the difference $w(z)-\hat{w}(z)$ satisfies an equation of the type (16) with $r(z)=O(z^{-2}\log z)$. Then applying Lemma 3, $w(z)-\hat{w}(z)\sim c_{1}+z^{-1}\log z$. With some help of Mathematica we arrived at

[TABLE]

From this and (21) for ${\rm Var}(N_{\theta^{*}}(z))=w(z)-u^{2}(z)$ we obtain

[TABLE]

with $c_{2}:=c_{1}-(c^{*})^{2}-{1}/{36}$. In fact, the value of $c^{*}$ in (21) impacts $c_{1}$ but not $c_{2}$, because the latter is invariant under shifting $u(0)$.

For the general control functions, the variance is very sensitive to the behaviour of $\theta$. The convergence $\theta(z)\to\sqrt{1/2}$ alone does not even ensure that $O(z)$ is the right order for ${\rm Var}(N_{\theta^{*}}(z))$. If (24) holds we have the asymptotics

[TABLE]

9. CLT for the number of jumps In this section we denote $N(z)$ the number of jumps of $Z|z$ with some control function satisfying

[TABLE]

Denote $J_{z}$ the size of the generic gap having the right endpoint $z$, with density

[TABLE]

and let $D_{z}$ be the size of the generic drift interval with survival function

[TABLE]

The size of the generic cycle with the right endpoint $z$ can be written as

[TABLE]

where $D_{z}$ and the family of variables $J_{\cdot}$ are independent, and we set $J_{0}=0$.

For large $z$, the expected values of $J_{z}$ and $D_{z}$ are about equal, suggesting that about a half of $[0,z]$ is covered by drift and another half is skipped by jumps. This resembles the behaviour of the stationary (and, as seen from [6], also of the optimal) selection process in the planar Poisson setting, where the balance is kept on two scales.

It is useful to see how the mean sizes of gaps and drift intervals depend on $\theta=\theta(z)$:

[TABLE]

The leading terms match for $\theta$ as in (26), in which case the mean size of a cycle is

[TABLE]

regardless of the $O(z^{-1})$ term in (26). This expansion explains why the second term in (23) is $O(\log z)$ (but falls short of explaining the coefficient $-1/6$), and why the suboptimal strategy in Theorem 5 is $O(1)$ from the optimum.

From the convergence of parameters (26) it is clear that as $z\to\infty$

[TABLE]

and, observing the joint convergence of $(D_{z},J_{z-\,\cdot})$, also that

[TABLE]

where $U\overset{{\rm d}}{=}{\rm Uniform}[0,1]$ and $E\overset{{\rm d}}{=}{\rm Exponential}(1)$ are independent.

The weak convergence (28) of cycle sizes suggests that the behaviour of $N(z)$ for large $z$ can be deduced from that of a renewal process with the generic step

[TABLE]

which has moments

[TABLE]

Specifically, for the renewal process $R(z):=\max\{n:H_{1}+\dots+H_{n}\leq z\}$, with $H_{j}$’s being i.i.d. replicas of $H$, we have the familiar CLT

[TABLE]

and one can expect that the same limit holds for $N(z)$. This line should be pursued with care, because local discrepancies may accumulate on the large scale and bias centring or even the type of the limit distribution. In the approach taken in the sequel, we amend some details of the method of stochastic comparison found in [9] (see a remark below). To that end, with initial state $z\to\infty$, we focus on the cycles that lie within some range $[\underline{z},z]$, where the truncation parameter $\underline{z}$ is properly chosen to warrant approximation of the whole process.

The asymptotics (26) implies that there exists a constant $c>0$ such that for all sufficiently large $\underline{z}$ the parameters can be bounded as

[TABLE]

uniformly in $z>\underline{z}$. Replacing the variable rate in (27) by constant yields the bounds

[TABLE]

where and henceforth $<_{\rm st}$ denotes the stochastic order. Observing that the survival function of $J_{z}$ is convex, we may bound the jump as

[TABLE]

whence from (3. The optimality equation)

[TABLE]

From these estimates follow stochastic bounds on the cycle size

[TABLE]

Setting the bounds (29) in terms of multiples of the same random variable $H$ is convenient in combination with the obvious scaling property: for $d>0$, $R(d\,\cdot)$ is the renewal process with the generic step $dH$. Let $N(z,\underline{z})$ be the number of cycles of $Z|z$, which fit completely within $[\underline{z},z]$. As in [9], from (29) we conclude that

[TABLE]

Letting $z\to\infty$ then $\underline{z}\to\infty$, and appealing to $R(z)/z\to\mu^{-1}$ a.s., (30) implies a weak law of large numbers for $N(z)$,

[TABLE]

We aim next to show the CLT for $N(z)$, that is

[TABLE]

To that end, we choose $\underline{z}=\omega\sqrt{z}$, where $\omega>0$ is a large parameter. Start with splitting

[TABLE]

where $N(z)-N(z,\underline{z})$ counts the cycles that start in $[0,\underline{z}]$; this component is annihilated by the scaling, since by (31)

[TABLE]

and the same is true with $\sqrt{\underline{z}}$ replaced by bigger $\sqrt{z}$. For the leading contribution due to $N(z,\underline{z})$ we obtain using dominance (30) and the CLT for $R(z)$

[TABLE]

as $z\to\infty$. Letting $\omega\to\infty$

[TABLE]

The opposite inequality is derived similarly. Hence (32) is proved.

Remark The renewal-type approximation for decreasing Markov chains on $\mathbb{N}$, using stochastic comparison appeared in [9]. However, their Theorem 4.1 on the normal limit for the absorption time fails without additional assumptions on the quality of convergence of the step distribution. For instance, if the decrement in position $z>8$ assumes values $1$ and $2$ with probabilities $1/2\pm 1/\log z$, the mean absorption time is asymptotic to $2z/3$, with the remainder being strictly of the order $z/\log z$, therefore not annihilated by the $\sqrt{z}$ scaling. The error in [9] appears on the bottom of page 996, where the truncation parameter ($m$, a counterpart of our $\underline{z}$) is assumed independent of the initial state. A recent paper [1], also concerned with the lattice setting, gives conditions on the rate of convergence of decrements in some probability metrics, to ensure the normal approximation of the absorption time.

Remark It is of interest to look at the properties of the random set $\cal Z$ which, intuitively, describes an infinite selection process. This limit object can be interpreted in the spirit of the boundary theory of Markov processes: the state space $[0,\infty)$ has a one-point compactification (the entrance Martin boundary) approached as the initial state of $Z|z$ tends to $\infty$. Applying the coupling argument as in Lemma 2 one can show that, at large distance from the origin, $\cal Z$ behaves similarly to a stationary alternating renewal process, with uniformly distributed gaps and exponential drift intervals. The coverage probability and the occupation measure satisfy $p(z)\to 1/2$ and $U([0,z])\sim\sqrt{2}z$, $z\to\infty$. There have been some related work on Markov processes on the real line which at distance from the origin behave similarly to renewal processes [12], but adapting existing results to our problem would require reverting the direction of time.

10. Summary We summarise our findings in terms of the original problem. Let $L_{\varphi}(t)$ be the length of increasing subsequence selected by a self-similar strategy with the acceptance window of the form (5).

Theorem 6**.**

1. (a)

The optimal strategy has the acceptance window of the form (5) with

[TABLE]

and outputs an increasing subsequence with expected length

[TABLE]

and variance

[TABLE] 2. (b)

The strategy with $\varphi_{0}(t):=\sqrt{{2}/{t}}\,\,\wedge 1$ outputs an increasing subsequence with the expected length

[TABLE]

and variance

[TABLE] 3. (d)

If $\varphi(t)\sim\sqrt{2/t}+O(t^{-1})$ then a central limit theorem holds:

[TABLE]

The instance of part (d) for the optimal strategy was proved in [6]; this can be compared with the distributional limit (4) for the stationary strategy.

Bruss and Delbaen [6] used concavity of $v$ to prove the bounds

[TABLE]

(for $t$ no too small), where $v(\beta)=2$. For large $t$, the logarithmic term in the lower bound has coefficient $-1/36$ (as is seen from (a)) and in the upper bound at least $0.55$ (as can be shown by estimating $\beta$). These bounds can be compared with the coefficient $1/72$ in part (a). Remark The version of the problem with a fixed number of observations $n$ is more complex, because the time of observation $m$ and the running maximum $y$ cannot be aggregated in a single state variable [2, 16]. Nevertheless, one can expect that the value function is well approximable by a function of $(n-m+1)(1-y)$, hence an analogue of self-similar strategy in Theorem 6 (b), that is the strategy with acceptance condition

[TABLE]

is close to optimality. Arlotto et al. [2] employed (1) and a de-Poissonisation argument to show that, indeed, the strategy given by (33) is within $O(\log n)$ from the optimum for $n$ large. Extending the methods of the present paper, the latter result has been strengthened recently in [17] : the expected length of subsequence selected with acceptance window (33) is $\sqrt{2n}-(\log n)/12+O(1)$, and this is within $O(1)$ from the optimum.