Online Revenue Maximization for Server Pricing

TL;DR

This paper develops an online posted-price mechanism for server resource pricing that maximizes revenue in a stochastic setting with unknown distributions, ensuring truthfulness and efficiency.

Contribution

It introduces a computationally efficient, revenue-optimal posted-price mechanism for online server pricing under uncertainty, with provable near-optimality from limited samples.

Findings

The mechanism achieves revenue optimality in expectation and retrospectively.

A polynomial number of samples suffices for near-optimal pricing.

Prices are deterministic and depend only on interval length and server availability.

Abstract

Efficient and truthful mechanisms to price resources on remote servers/machines has been the subject of much work in recent years due to the importance of the cloud market. This paper considers revenue maximization in the online stochastic setting with non-preemptive jobs and a unit capacity server. One agent/job arrives at every time step, with parameters drawn from an underlying unknown distribution. We design a posted-price mechanism which can be efficiently computed, and is revenue-optimal in expectation and in retrospect, up to additive error. The prices are posted prior to learning the agent's type, and the computed pricing scheme is deterministic, depending only on the length of the allotted time interval and on the earliest time the server is available. If the distribution of agent's type is only learned from observing the jobs that are executed, we prove that a polynomial number of samples is sufficient to obtain a near-optimal truthful pricing strategy.

Equations

$${\mathsf{CmlRev}}_T(\mathcal{X},\pi ):={\sum }_{t=0}^T{\mathsf{Rev}}_t(\mathcal{X},\pi ).$$

$$U_t^{\pi }(s)={\mathbb{E}}_{{\mathcal{X}}_{\ge }t}\left[{\sum }_{i=t}^T{\mathsf{Rev}}_i(\pi ,\mathcal{X})|S_t=s\right],$$

$$\mathbb{P}[s_{t+1}=s_t+\ell -1]=\{\begin{array}{ll}\mathbb{P}\left[L_t=\ell ,V_t\ge {\pi }_t(s_t,\ell ),D_t\ge s_t+\ell \right]& \text{ if }\ell \ge 1\\ 1-{\sum }_{k\ge 0}\mathbb{P}[s_{t+1}=s_t+k]& \text{ if }\ell =0\end{array}$$

$$\mathbb{P}[(s,\ell )$$

$$R((s,\ell )$$

$$U_t^{\pi }(s)=E_L\left[u_t^{\pi }(s,L)\right],\forall t\in T,s\in \text{SS},$$

$$O\left({\mathbb{K}}^3{\log }_{\gamma }\left(\frac{\epsilon (1-\gamma )}{\mathbb{V}}\right)\right)\le O\left(\frac{{\mathbb{K}}^3}{1-\gamma }\ln \left(\frac{\mathbb{V}}{\epsilon (1-\gamma )}\right)\right)$$

$$R_i^{\pi }:=E\left[{\mathsf{CmlRev}}_T(\pi ,\mathcal{X})|{\mathcal{X}}_1,\dots ,{\mathcal{X}}_i\right]$$

$$R_i^{\pi }$$

$$=\left({\sum }_{t=0}^i{\mathsf{Rev}}_t(\pi ,{\mathcal{X}}_{\le t})\right)+U_{i+1}^{\pi }(S_{i+1}(\pi ,{\mathcal{X}}_{\le i}))$$

$$\left|{\mathsf{CmlRev}}_T(\mathcal{X},\pi )-E_{{\mathcal{X}}^{\prime }}\left[{\mathsf{CmlRev}}_T({\mathcal{X}}^{\prime },\pi )\right]\right|\le \mathbb{V}\cdot \sqrt{2\log (\text{nicefrac}2\delta )T}.$$

$$\left|{\mathsf{CmlRev}}_{\infty }(\mathcal{X},\pi )-E_{{\mathcal{X}}^{\prime }}\left[{\mathsf{CmlRev}}_T({\mathcal{X}}^{\prime },\pi )\right]\right|\le \mathbb{V}\cdot \sqrt{2\log (\text{nicefrac}2\delta )/(1-{\gamma }^2)}.$$

$$\left|\mathbb{P}(\hat{L}=\ell ,\hat{V}\ge v,\hat{D}\ge s)-\mathbb{P}(L=\ell ,V\ge v,D\ge s)\right|<\epsilon \forall s\in S,\ell \in \mathcal{L}\text{ and }v\in \mathcal{V}.$$

$${U^{\pi}_{t}(s)}:=\sum_{\ell\in\mathcal{L}}\mathbb{P}[L=\ell]\Big{(}\bm{p}^{\ell}_{t,s}\big{(}\pi_{t}(s,\ell)+{U^{\pi}_{t+1}(s+\ell-1)}\big{)}+(1-\bm{p}^{\ell}_{t,s}){U^{\pi}_{t+1}(s-1)}\Big{)},$$

$$\Big{|}\tfrac{1}{n}\textstyle\sum_{k=1}^{n}\mathbb{I}[L_{k}=\ell,V_{k}\geq v,D_{k}\geq s]-\mathbb{P}[L=\ell,V\geq v,D\geq s]\Big{|}\leq\sqrt{\tfrac{\log(\nicefrac{{2}}{{\delta}})}{2n}}$$

$${\mathsf{ECRev}}_T(\pi |Q):=E_{\mathcal{X}\sim Q}\left[{\mathsf{CmlRev}}_T(\mathcal{X},\pi )\right]$$

$${\mathsf{CmlRev}}_T(\mathcal{X},\hat{\pi })\ge {\mathsf{CmlRev}}_T(\mathcal{X},\pi )-2\mathbb{V}\sqrt{2\log (\text{nicefrac}8\delta )(T+1)}-\epsilon$$

$${\mathsf{CmlRev}}_T(\mathcal{X},\hat{\pi })\ge {\mathsf{CmlRev}}_T(\mathcal{X},\pi )-2\mathbb{V}\sqrt{2\log (\text{nicefrac}8\delta )(T+1)}-\epsilon -\eta T$$

$${\mathsf{CmlRev}}_T(\mathcal{X},\hat{\pi })$$

$$\ge {\mathsf{ECRev}}_T(\hat{\pi }|\hat{Q})-\mathbb{V}\sqrt{2\log (8/\delta )(T+1)}-\epsilon /2$$

$$\ge {\mathsf{ECRev}}_T({\pi }^{\ast }|\hat{Q})-\mathbb{V}\sqrt{2\log (8/\delta )(T+1)}-\epsilon /2$$

$$\ge {\mathsf{ECRev}}_T({\pi }^{\ast }|Q)-\mathbb{V}\sqrt{2\log (8/\delta )(T+1)}-\epsilon$$

$$\ge {\mathsf{ECRev}}_T(\pi |Q)-\mathbb{V}\sqrt{2\log (8/\delta )(T+1)}-\epsilon$$

$$\ge {\mathsf{CmlRev}}_T(\mathcal{X},\pi )-2\mathbb{V}\sqrt{2\log (8/\delta )(T+1)}-\epsilon$$

$${\mathsf{CmlRev}}_{\infty }(\mathcal{X},\hat{\pi })\ge {\mathsf{CmlRev}}_{\infty }(\mathcal{X},\pi )-2\mathbb{V}\sqrt{2\log (\text{nicefrac}8\delta )/(1-{\gamma }^2)}-2\epsilon$$

$${\mathsf{CmlRev}}_{\infty }(\mathcal{X},\hat{\pi })\ge {\mathsf{CmlRev}}_{\infty }(\mathcal{X},\pi )-2\mathbb{V}\sqrt{2\log (\text{nicefrac}8\delta )/(1-{\gamma }^2)}-2\epsilon -\eta /(1-\gamma )$$

$$p_y{p}_{y+k}\ge p_{y-1}{p}_{y+k+1}\forall y,k\in \mathbb{Z},k\ge 0.$$

$$P(y)P(y)$$

$$\ge P(y-1)P(y)+\sum _{-\infty }^y{p}_{k-1}{p}_{y+1}=P(y-1)P(y+1)$$

$$\mathbb{P}[V\ge \mu ,D\ge s|L=\ell ]=\mathbb{P}[V\ge \mu |D\ge s,L=\ell ]\cdot \mathbb{P}[D\ge s|L=\ell ].$$

$$\log \bar{F}(x-\gamma )-\log \bar{F}(x^{\prime }-\gamma )\ge \log \bar{F}(x-{\gamma }^{\prime })-\log \bar{F}(x^{\prime }-{\gamma }^{\prime })$$

$$\log \bar{F}(x/\gamma )-\log \bar{F}(x^{\prime }/\gamma )$$

$$\ge \log \bar{F}(x/{\gamma }^{\prime })-\log \bar{F}(x^{\prime }/{\gamma }^{\prime })$$

$$U_t^{\pi }(s)=E_L\left[u_t^{\pi }(s,L)\right],\forall t\in T,s\in \text{SS},$$

$$E_L\left[u_t^{\pi }(s,L)\right]$$

Full text

1 Introduction

Designing mechanisms for a desired outcome with strategic and selfish agents is an extensively studied problem in economics, with classical work by Myerson [1], and Vickrey-Clarke-Groves [2], for truthful mechanisms. The advent of online interaction and e-commerce has added an efficiency constraint on the mechanisms, going so far as to prioritize computational efficiency over classical objectives: e.g. choosing simple approximate mechanisms when optimal mechanisms are computationally difficult, or impossible. Beginning with Nisan and Ronen [3], the theoretical computer science community has contributed greatly to the field, in both fundamental problems and specific applications. These include designing truthful mechanisms for the maximization of welfare and revenue, and has also focused on learning distributions of agent types, menu complexity, and dynamic mechanisms (e.g., [4, 5].)

We consider this question in the setting of selling computational resources on remote servers or machines (cf. [6, 7].) This is arguably one of the fastest growing markets on the Internet. The goods (resources) are assigned non-preemptively and thus have strong complementarities. Furthermore, since the supply (server capacity) is limited, any mechanism trades immediate revenue for future supply. Finally, mechanisms must be incentive-compatible, as non-truthful, strategic, behaviour from the agents can skew the performance of a mechanism from its theoretical guarantees. This leads us to the following question:

Question.

Can we design an efficient, truthful, and revenue-maximizing mechanism to sell time-slots non-preemptively on a single server?

We design a posted-price mechanism which maximizes expected revenue up to additive error, for agents/buyers arriving online, with parameters of value, length and maximum delay, drawn from an underlying distribution.

Three key aspects distinguish our problem from standard online scheduling: (i) In our setting, as time progresses, the server clears up, allowing longer jobs to be scheduled in the future if no smaller jobs are scheduled until then. (ii) Scheduling the jobs is not exclusively to the discretion of the mechanism designer, but must also be desired by the job itself, while also producing sufficient revenue. (iii) As the mechanism designer, we do not have access to job parameters in an incentive-compatible way before deciding on a posted price menu. These three features lie at the core of the difficulty of our problem. Our focus will be on devising online mechanisms in the Bayesian setting.

In our online model, time on the server is discrete. At every time step, an agent arrives on the server, with a value $V$, length requirement $L$, and maximum delay $D$. These parameters are drawn from a common distribution, i.i.d. across jobs. The job wishes to be scheduled for at least $L$ consecutive time slots, no more than $D$ time units after its arrival, and wishes to pay no more than $V$. Jobs are assumed to have quasi-linear utility in money, and so prefer the least-price interval within their constraints. The mechanism designer never learns the parameters of the job. Instead, she posts a price menu of (length,price) pairs, and the minimum available delay $s$. The job accepts to be scheduled so long as $D\geq s$, and there is some (length,price) pair in the menu of length at least $L$ and price at most $V$. We note that the pricing scheme can be dynamic, changing through time. If, at time epoch $t$, an agent chooses option $(\ell,\pi_{\ell})$, then she pays $\pi_{\ell}$ and her job will be allocated to the interval $[t+s,t+s+\ell]$. She will choose the option which minimizes $\pi_{\ell}$. Throughout this paper we assume that the random variables $L,V,D$ are discrete, and have finite support, unless specified differently.

1.1 Summary of Our Results

We model the problem of finding a revenue maximizing pricing strategy as a Markov Decision Process (MDP). Given a price menu (length,price) and a state (minimum available delay) $s$ at time $t$, the probability of transition to any other state at time $t+1$ is obtained from the distribution of the job’s parameters. The revenue maximizing pricing strategy can be efficiently computed via backwards induction. We also present, in Appendix C.2, an approximation scheme in the case where $V$ is a continuous random variable. 2. 2.

We prove that the optimal pricing strategy is monotone in length under a distributional assumption, which we show is satisfied when the jobs’ valuation follows a log-concave distribution, parametrized by length. Recall that log-concave distributions are exactly those which have a monotone hazard rate. This implies the existence of an optimal pricing mechanism which ensures truthfulness in the finite horizon setting when the distributions are known. In Appendix C.1, this is extended to the infinite discounted horizon setting, incurring a small additive error. We also demonstrate good concentration bounds of the revenue obtained by the optimal truthful posted price strategy. 3. 3.

We finally investigate the robustness of the pricing strategy. We first show that a near optimal solution is still obtained when the distribution is known with a certain degree of uncertainty. We complement this result by analyzing the performances of the proposed pricing strategy when the distribution is only known from samples collected through the observations of the agents’ decisions. We provide a truthful posted price $\varepsilon$-approximate mechanism if the number of samples is polynomial in $1/\varepsilon$ and the size of the support of the distribution.

1.2 Related Work

Much recent work has focused on designing efficient mechanisms for pricing cloud resources. Chawla et al. [8] recently studied “time-of-use” pricing mechanisms, to match demand to supply with deadlines and online arrivals. Their result assumes large-capacity servers, and seeks to maximize welfare in a setting in which the jobs arriving over time are not i.i.d.. [9] provides a mechanism for preemptive scheduling with deadlines, maximizing the total value of completed jobs. Another possible objective for the design of incentive-compatible scheduling mechanisms is the total value of completed jobs, which have release times and deadlines. [10] solves this problem in an online setting, while [11], in the offline setting for parallel machines, and [12], in the online competitive setting with uncertain supply. [13] focuses on social welfare maximization for non-preemptive scheduling on multiple servers, and obtains a constant competitive ratio as the number of servers increases. Our work differs from these by considering revenue maximization and stochastic job types which are i.i.d. over time. [14] addresses computing a price menu for revenue maximization with different machines. Finally, [7] proposes a system architecture for scheduling and pricing in cloud computing.

Posted price mechanisms (PPM) have been introduced by [15] and have gained attention due to their simplicity, robustness to collusion, and their ease of implementation in practice. One of the first theoretical results concerning PPM’s is an asymptotic comparison to classical single-parameter mechanisms [16]. They were later studied by [17] for the objective of revenue maximization, and further strengthened by [18] and [19]. [20] shows that sequential PPM’s can $\nicefrac{{1}}{{2}}$-approximate social welfare for XOS valuation functions, if the price for an item is equal to the expected contribution of the item to the social welfare.

Sample complexity for revenue maximization was recently been studied in [5] showing that a polynomially many of samples suffice to obtain near optimal Bayesian auction mechanisms. An approach based on statistical learning that allows to learn mechanisms with expected revenue arbitrarily close to optimal from a polynomial number of samples has been proposed in [21]. The problems of learning simple auctions from samples has been studied in [22].

1.3 Structure of the Paper

In Section 2 we describe the model of the problem as a Markov Decision Process. In Section 3 we present an efficient algorithm for computing optimal policies for the finite time horizon given full knowledge of the distribution of the jobs’ paramethers. This is extended to other settings in Appendix C. In Section 3.3, we demonstrate that the optimal policy is monotone and in  Section 3.4 we describe the concentration bounds on the revenue of a pricing policy.  Section 4.2 gives the learning algorithm and error bounds for computing the pricing policies with only (partial) sample access to the job distribution. In Finally, Section 4.3 and Section 5 are devoted to describing and summarizing the final result and future directions of research.

Proof details are provided in Appendix B.

2 Model

Notation.

In what follows, the variables $t$, $\ell$ or $L$, $v$ or $V$, and $d$ or $D$ are reserved for describing the parameters of a job that wishes to be scheduled. Respectively, they represent the arrival time $t$, required length $\ell$, value $v$, and maximum allowed delay $d$. The lowercase variables represent fixed values, whereas the uppercase represent random variables. Script-uppercase letters $\mathcal{L},\mathcal{V},\mathcal{D}$ represent the supports of the distributions on $L$, $V$, and $D$, respectively; and the bold-uppercase letters $\mathbb{L},\mathbb{V},\mathbb{D}$ represent the maximum values in these respective sets. Finally, $\pi$ is reserved for pricing policy, whereas $p$ is reserved for probabilities.

Single-Machine, Non-Preemptive, Job Scheduling.

A sequence of random jobs wish to be scheduled on a server, non-preemptively, for a sufficiently low price, within a time constraint. Formally, at every time step $t$, a single job with parameters $(L,V,D)$ is drawn from an underlying distribution $Q$ over the space $\mathcal{L}\times\mathcal{V}\times\mathcal{D}$. It wishes to be scheduled for a price $\pi\leq V$ in an interval $[a,b]$ such that $a-t\leq D$ and $b-a\geq L$.

Price Menus.

Our goal is to design a take-it-or-leave-it, posted-price mechanism which maximizes expected revenue. At each time period, the mechanism posts a “price menu” and an earliest-available-time $s_{t}$, indicating that times $t$ through $t+s_{t}-1$ have already been scheduled. ($s_{t}$ will henceforth be referred to as the state of the server.) We let $\SS:=\{0,\,\dotsc,\,\mathbb{D}+\mathbb{L}\}$ to be the set of all possible states. The state of the server at a given time $t$ is naturally a random variable which depends on the earlier jobs and on the adopted policy $\pi$. As before, we will denote with $s$ or $s_{t}$ the fixed value, and with $S$ or $S_{t}$ the corresponding random variable. The price menu will be given by the function $\pi:[T]\times\SS\times\mathcal{L}\to\mathbb{R}$, i.e., if we are a time $t$ and the server is in state $s_{t}$, then the prices are set according to $\pi_{t}(s_{t},\cdot):\mathcal{L}\to\mathbb{R}.$ The reported pair $(\pi_{t}(s_{t},\cdot),s_{t})$ is computed by the scheduler’s strategy, which we determine in this paper. Once this is posted, a job $(L,V,D)$ is then sampled i.i.d. from the underlying distribution $Q$.

If $V\geq\pi_{t}(s_{t},\ell)$ for some $\ell\geq L$, and $D\geq s_{t}$, then the job accepts the schedule, and reports the length $\ell\geq L$ which minimize price. Otherwise, the job reports $\ell=0$ and is not scheduled. To guarantee truthfulness, it suffices to have $\pi_{t}(s,\cdot)$ be monotonically non-decreasing for every state $s$: the agent would not want a longer interval since it costs more, and would not want one of the shorter intervals since they cannot run the job. It should be clear that the mechanism’s strategy is to always report monotone non-decreasing prices, as a decrease in the price menu will only cause more utilization of the server, without accruing more revenue. The main technical challenge in this paper, then, is to show that under some assumptions, the optimal strategy is monotone non-decreasing, and efficiently computable.

Revenue Objective.

Revenue can be measured in either a finite or an infinite discounted horizon. In the former (finite) case, only $T$ time periods will occur, and we seek to maximize the expected sum of revenue over these periods. In the infinite-horizon setting, future revenue is discounted, at an exponentially decaying rate. Formally, revenue at time $t$ is worth a $\gamma^{t}$ fraction of revenue at time 0, for some fixed $\gamma<1$. See Appendix C.1. Recall that the job parameters are drawn independently at random from the underlying distribution, so the scheduler can only base their “price menu” on the state of the system and the current time. Thus, the only realistic strategy is to fix a state-and-time-dependent pricing policy $\pi:[T]\times\SS\times\mathcal{L}\to\mathbb{R}$, “$\pi_{t}(s,\ell)$”, where $[T]:=\{0,\,1,\,\dotsc,\,T\}$.

Let $\mathcal{X}=\{\mathcal{X}_{1}:=(1,L_{1},V_{1},D_{1}),\,\mathcal{X}_{2}:=(2,L_{2},V_{2},D_{2}),\,\mathcal{X}_{3},\,\dotsc\}$ be the random sequence of jobs arriving, sampled i.i.d. from the underlying distribution. Let $\pi:[T]\times\SS\times\mathcal{L}\to\mathbb{R}$ be the pricing policy. We denote as $\mathsf{Rev}_{t}(\mathcal{X},\pi)$ the revenue earned at time $t$ with policy $\pi$ and sequence $\mathcal{X}$. If $\mathcal{X}_{t}$ does not buy, then $\mathsf{Rev}_{t}(\mathcal{X},\pi)=0$, and otherwise, it is equal to $\pi_{t}(s_{t},L_{t})$. We denote as $\mathsf{CmlRev}_{T}$ the total (cumulative) revenue earned over the $T$ periods. Thus,

[TABLE]

We will also need the expected-future-revenue, given a current time and server state, which we will denote as follows:

[TABLE]

The subscript of the expectation $\mathcal{X}_{\geq t}$ denotes that we consider only jobs arriving from time $t$ onward. Our objective is to find the pricing policy $\pi$ which maximizes ${U^{\pi}_{0}(s=0)}$. Call this $\pi^{*}$, and denote the expected revenue under $\pi^{*}$ as ${U^{*}_{t}(\cdot)}$.

3 Bayes-optimal Strategies for Sever Pricing

In this section we seek to compute an optimal monotone pricing policy $\pi:[T]\times\SS\times\mathcal{L}\to\mathbb{R}$ which maximizes revenue in expectation over $T$ jobs sampled i.i.d. from an underlying known distribution $Q$. This is extended to the infinite-horizon, discounted, setting in Appendix C.1.

We first model the problem of maximizing the revenue in online server pricing as a Markov Decision Process that admits an efficiently-computable, optimal pricing strategy. The main contribution of this section is to show that, for a natural assumption on the distribution $Q$, the optimal policy is monotone. We recall that this allows us to derive truthful Bayes-optimal mechanisms.

3.1 Markov Decision Processes.

We show that the theory of Markov Decision Processes is well suited to model our problem. A Markov Decision Process is, in its essence, a Markov Chain whose transition probabilities depend on the action chosen at each state, and where to each transition is assigned a reward. A policy is then a function $\pi$ mapping states to actions. In our setting, the states are the states of the system outlined in Section 2 (i.e., the possible delays before the earliest available time on the server), and the actions are the “price menus.” At every state $s$, a job of a random length arrives, and with some probability, chooses to be scheduled, given the choice of prices. The next state is either $\max\{s-1,0\}$, if the job does not choose to be scheduled (since we have moved forward in time), or $s+\ell-1$, if a job of length $\ell$ is scheduled, since we have occupied $\ell$ more units. The transition probabilities depend on the distribution of job lengths, and the probability that a job accepts to be scheduled given the pricing policy (action). Formally,

[TABLE]

(Transitions to state “$-1$” should be read as transitions to state “$\,0\!$”.)  Note that a job of length $\ell$ may choose to purchase an interval of length greater than $\ell$, which would render these transition probabilities incorrect. However, this may only happen if the larger interval is more affordable. It is therefore in the scheduler’s interest to guarantee that $\pi_{t}(s,\cdot)$ in monotone non-decreasing in $\ell$, which incentivizes truthfulness, since this increases the amount of server-time available, without affecting revenue. Thus we restrict ourselves to this case.

It remains to define the transition rewards. They are simply the revenue earned. Formally, a transition from state $s_{t}$ to $s_{t}+\ell-1$ incurs a reward of $\pi_{t}(s,\ell)$, whereas a transition from state $s_{t}$ to $s_{t}-1$ incurs 0 reward. We wish to compute a policy $\pi$ in such a way as to maximize the expected cumulative revenue, given as the (possibly discounted) sum of all transition rewards in expectation.

3.2 Solving for the Optimal Policy with Distributional Knowledge

In this section, we present a modified MDP whose optimal policies can be efficiently computed, and show that these policies are optimal for the original MDP. In this section, we assume that the mechanism designer is given access to the underlying distribution $Q$. However, in the following sections, we will show that if the distribution $Q$ is estimated from samples, then solving for the MDP on this estimated distribution is sufficient to ensure sufficiently good revenue guarantees.

Since the problem has been modelled as a Markov Decision Process (MDP), we may rely on the wealth of literature available on MDP solutions, in particular we will leverage the backwards induction algorithm (BIA) of [23] Section 4.5, included in Appendix B as Algorithm 1. We will however need to ensure that this standard algorithm (i) runs efficiently, and (ii) returns a monotone pricing policy.

Note that past prices do not contribute to future revenue insofar as the current state remains unchanged. Thus, to compute optimal current prices, we need only know the current state and expected future revenue. This allows us to use the BIA. The idea is to compute the optimal time-dependent policy, and the incurred expected reward, for shorter horizons, then use this to recursively compute the optimal policies for longer horizons.

The total runtime of the BIA is $O(T|\mathcal{S}||\mathcal{A}|)$, where $\mathcal{S}$ and $\mathcal{A}$ denote the action and state spaces, respectively. Note that the dependence on $T$ is unavoidable, since any optimal policy must be time-dependent. Recall that $\mathbb{L}$ and $\mathbb{D}$ denote the maximum values that $L$ and $D$ can take, respectively, and $\mathcal{V}$ is the set of possible values that $V$ can take. Denote $\mathbb{K}:=\max\{\mathbb{D}+\mathbb{L},|\mathcal{V}|\}$. If we define the action space naïvely, we have $|\mathcal{S}|=\mathbb{D}+\mathbb{L}\leq\mathbb{K}$, and $|\mathcal{A}|\leq\mathbb{K}^{\mathbb{L}}$. Thus, a naïve definition of the MDP bounds the runtime at $\mathbb{K}^{O(\mathbb{K})}$, which is far from efficient. Requiring monotonocity only affects lower-order terms.

Modified MDP.

To avoid this exponential dependence, we can be a little more clever about the definition of the state space: instead of states being the possible server states, we define our state space as possible (state, length) pairs. Thus, when the MDP is in state $(s,\ell)$, the server is in state $s$, and a job of length $\ell$ has been sampled from the distribution. Our action-space then is simply the possible values of $\pi_{t}(s,\ell)$, and the transition probabilities and rewards become:

[TABLE]

Therefore, we get $|\mathcal{S}|=(\mathbb{D}+\mathbb{L})\cdot\mathbb{L}\leq\mathbb{K}^{2}$, and $|\mathcal{A}|\leq\mathbb{K}$. Thus, the runtime of the algorithm becomes $O(T\mathbb{K}^{3})$. A full description of the procedure is given in Appendix B as Algorithm 2. It remains to prove that it is correct. We begin by claiming that these two MDPs are equivalent in the following sense:

Lemma 1**.**

For any fixed pricing policy $\pi:[T]\times\SS\times\mathcal{L}\to\mathbb{R}$,

[TABLE]

where the ${U^{\pi}_{t}(\cdot)}$’s are as in (2), and the ${u^{\pi}_{t}(\cdot,\cdot)}$’s are from the modified MDP.

(See Appendix B for a proof.) This lemma, however, does not suffice on its own, as agents may behave strategically by over-reporting their length, if the prices are not increasing. This would alter the transition probabilities, breaking the analysis. We will see that under a mild assumption, this can not happen, as the optimal policy for non-strategic agents will be monotone, and therefore truthful.

3.3 Monotonicity of the Optimal Pricing Policies

Recall that the solution of the more efficient MDP formulation is only correct if we can show that it is always monotone without considering the strategic behaviour of agents, ensuring incentive-compatibility of the optimum.

An optimal monotone strategy cannot be obtained for all the distributions on $L,V,$ and $D$. As an example, for any distribution where a job’s value is a deterministic function of their length, the optimal policy is to price-discriminate by length. If this function is not monotone, the optimum won’t be either. To this end, we introduce the following assumption, which we will discuss below, and which will imply monotonicity of the pricing policy.

Assumption 1.

The quantity $\tfrac{\mathbb{P}[V\geq{\mu}^{\prime},D\geq s|L=\ell]}{\mathbb{P}[V\geq{\mu},D\geq s|L=\ell]}$ is monotone non-decreasing as $\ell$ grows, for any state $s$ and $0\leq{\mu}<{\mu}^{\prime}$ fixed.

This is not a natural, or immediately intuitive assumption. However, we will show that it is satisfied if the valuation of jobs follows a log-concave distribution which is parametrized by the job’s length, and where the valuation is (informally) positively correlated with this length. Log-concave distributions are also commonly referred to as distributions possessing a monotone hazard rate, and it is common practice in economic settings to require this property of the agent valuations.

Lemma 2**.**

Let, $V_{\ell}^{s}$ denote the marginal r.v. $V$ conditioned on $L=\ell$ and $D\geq s$. Let $Z$ be a continuously-supported random variable, and $\gamma_{1}^{s}\leq\gamma_{2}^{s}\leq\dotsm\in\mathbb{R}$. If $V_{\ell}^{s}$ is distributed like $\gamma_{\ell}^{s}\cdot Z$, $\left\lfloor\gamma_{\ell}^{s}\cdot Z\right\rfloor$, $Z+\gamma_{\ell}^{s}$, or $\left\lfloor Z+\gamma_{\ell}^{s}\right\rfloor$, then Assumption 1 is satisfied if $Z$ is log-concave, or if the $\gamma$’s are independent of $\ell$.

A discussion of log-concave random variables and a proof of this fact is given in Appendix A. Many standard (discrete) distributions are (discrete) log-concave random variables, including the uniform, Gaussian, logistic, exponential, Poisson, binomial, etc. These can be proved to be log-concave from the discussion in Appendix A. In the above, the $\gamma$ terms represent a notion of spread or shifting, parametrized by the length, indicating some amount of positive correlation.

It remains to show price monotonicity under the above assumption. First, we begin with the following, which holds for arbitrary distributions.

Lemma 3**.**

Let ${U^{*}_{t}(s)}$ be the expected future revenue earned starting at time $t$ in state $s$, for the optimal policy computed by Algorithm 2. Then the function $s\mapsto{U^{*}_{t}(s)}$ is monotone non-increasing in $s$ for any $t$ fixed.

See Appendix B for the proof. This lemma ensures that over-selling time on the server can only hurt the mechanism. This allows us to conclude

Lemma 4**.**

If the distribution on job parameters satisfies the above assumption, then for all $\ell,s,t$, we have $\pi^{*}_{t}(s,\ell)\leq\pi^{*}_{t}(s,\ell+1)$.

Sketch..

A full proof may be found in Appendix B. The idea is to show that, for any price ${\mu}$ less than the optimum $\pi^{*}_{t}(s,\ell)$, the difference in revenue between charging ${\mu}$ and $\pi^{*}_{t}(s,\ell)$ to jobs of length $\ell$ is less than the difference in revenue between the same prices for jobs of length $\ell+1$. This is achieved by applying the assumption to recursive definition of future revenue, along with the previous lemma. Thus, we can conclude that the optimal price $\pi_{t}^{*}(s,\ell+1)$ must be greater than $\pi^{*}_{t}(s,\ell)$. ∎

With Lemma 4 and the results of Appendix C, we finally have:

Theorem 5**.**

The online server pricing problem admits an optimal monotone pricing strategy when the variables $L$, $V$, and $D$ satisfy assumption 1. Also,

In the finite horizon setting, when $\mathcal{V}$ is finitely supported, an exact optimum can be computed in time $O(T\mathbb{K}^{3})$. 2. 2.

In the infinite horizon setting, when $\mathcal{V}$ is finitely supported, for all $\varepsilon>0$, an $\varepsilon$-additive-approximate policy can be computed in time

[TABLE] 3. 3.

In the finite horizon setting, when $V$ is continuously supported, for all $\eta>0$, an $\eta T$-additive-approximate policy can be computed in time $O(T\mathbb{K}^{2}\mathbb{V}/\eta)$.

3.4 Concentration Bounds on Revenue for Online Scheduling

In this section, we show that the revenue of arbitrary policies concentrates around their mean. In particular it holds true for the optimal or approximately optimal strategies described above. This will also allow us to argue later that, if we have an estimate $\hat{Q}$ of $Q$, then execute Algorithm 2 given the distribution $\hat{Q}$, then the output policy will perform well with respect to $Q$, both in expectation, and with high probability. To show this concentration, we will consider the Doob or exposure martingale of the cumulative revenue function, introduced in Section 2. Define

[TABLE]

where the $\mathcal{X}_{i}$’s are jobs in the sequence $\mathcal{X}$ and the expected value is taken with respect to $\mathcal{X}_{i+1},\dots\mathcal{X}_{T}$. Thus, $R_{0}^{\pi}$ is the expected cumulative revenue, and $R_{T}^{\pi}$ is the random cumulative revenue. To formally describe this martingale sequence, we will introduce some notation, and formalize some previous notation. Recall that $\mathcal{X}_{1},\,\mathcal{X}_{2},\,\dotsc$ is a sequence of jobs sampled i.i.d. from an underlying distribution $Q$. Fix a pricing policy ${\pi:[T]\times\SS\times\mathcal{L}\to\mathbb{R}}$. Note that the state at time $t$ is a random variable depending on both the (deterministic) pricing policy and the (random) $\mathcal{X}_{1},\dots,\mathcal{X}_{t-1}$. We denote it $S_{t}(\pi,\mathcal{X})$, or $S_{t}$ for short. Formally, suppose $\mathcal{X}_{t}=(V_{t},L_{t},D_{t})$, then $S_{t+1}(\pi,\mathcal{X})=S_{t}(\pi,\mathcal{X})-1$ if either $V_{t}<\pi_{t}(S_{t},L_{t})$ or $D_{t}<S_{t}$, and otherwise $S_{t+1}(\pi,\mathcal{X})=S_{t}(\pi,\mathcal{X})+L_{t}-1$. Furthermore, let $\mathsf{Rev}_{t}(\pi,\mathcal{X})$ be equal to 0 in the first case above (the $t$-th job is not scheduled), and $\pi_{t}(S_{t},L_{t})$ otherwise. Thus, $S_{t}(\pi,\mathcal{X})$ and $\mathsf{Rev}_{t}(\pi,\mathcal{X})$ are functions of the random values $\mathcal{X}_{1},\,\dotsc,\,X_{t}$ for $\pi$ fixed. Note that $\mathsf{Rev}_{t}$ implicitly depends on $S_{t}$. Let $\mathcal{X}_{>i}:=(\mathcal{X}_{i+1},\mathcal{X}_{i+2},\ldots)$ and $\mathcal{X}_{\leq i}:=(\mathcal{X}_{1},\ldots\mathcal{X}_{i})$. Recalling that $\mathsf{CmlRev}_{T}(\mathcal{X},\pi)=\sum_{t=1}^{T}\mathsf{Rev}_{t}(\mathcal{X},\pi)$, we have

[TABLE]

We wish to show that $\mathsf{CmlRev}(\mathcal{X},\pi)$ concentrates around its mean. Since $R_{0}^{\pi}$ is the expected revenue due to $\pi$, and $R_{T}^{\pi}$ is the (random) revenue observed, it suffices to show $|R_{0}^{\pi}-R_{T}^{\pi}|$ is small, which we will do by applying Azuma’s inequality, after showing the bounded-differences property. This gives, see Appendix B.3 for details,

Theorem 6**.**

*Let $\mathcal{X}$ be a finite sequence of $T$ jobs sampled $i.i.d.$ from $Q$, and let $\pi$ be any monotone policy. Then, with probability $1-\delta$,

[TABLE]

*in the finite horizon, and in the infinite-horizon-discounted,

[TABLE]

In particular these results hold true for the (approximately) optimal pricing strategies of Theorem 5.

4 Robustness of Pricing with Approximate Distributional Knowledge

In this section, we show that results analogous to Theorems 5 and 6 may be obtained even in the case in which we do not have full knowledge of the distribution $Q$, but only an estimate $\hat{Q}$. We then show how to obtain a valid $\hat{Q}$ from samples.

4.1 Robustness of the pricing strategy

Let’s suppose that instead of knowing the exact distribution $Q=(D,L,V)$ of the jobs, we have only access to some estimate $\hat{Q}=(\hat{D},\hat{L},\hat{V})$ with the following property, for some $\varepsilon>0$:

[TABLE]

For the sake of brevity, we abuse notation and denote the condition in $(\ref{eq:robustness})$ as $|Q-\hat{Q}|<\varepsilon.$ Later, we will need to estimate the value $\mathbb{P}[L=\ell,\neg(V\geq v,D\geq s)]$, given $\hat{Q}$, that is the probability that the job has length $\ell$, but either cannot afford price $v$, or cannot be scheduled $s$ slots in the future. This is equal to \mathbb{P}[L=\ell]-\mathbb{P}[L=\ell,V\geq v,D\geq s]\.

The left-hand term is equal to $\mathbb{P}[{L=\ell,}{V\geq 0,}{D\geq 0]}$, and so we have access to both terms. The estimation error is additive, so the deviation is at most $2\varepsilon$.

Denote $\bm{p}^{\ell}_{t,s}:=\mathbb{P}[V\geq\pi^{t}(s,\ell),D\geq s|L=\ell]$, and recall

[TABLE]

the expected revenue from time $t$ onwards, conditioning on $S_{t}=s$. Let ${\hat{U}^{\pi}_{t}(\cdot)}$ be the same as ${U^{\pi}_{t}(\cdot)}$, but where the variables are distributed as $\hat{Q}$. As before, let ${U^{*}_{t}(\cdot)}$ be ${U^{\pi}_{t}(\cdot)}$ for $\pi=\pi^{*}$, the Bayes-optimal policy returned by Algorithm 2, and ${\hat{U}^{*}_{t}(\cdot)}$ defined similarly but with respect to $\hat{Q}$. We will show that ${\hat{U}^{*}_{t}(\cdot)}$ is a good estimate for ${U^{*}_{t}(\cdot)}$.

Lemma 7**.**

Let $Q$, and $\hat{Q}$ such that $|Q-\hat{Q}|<\varepsilon$.

In the finite horizon, $|{U^{*}_{t}(s)}-{\hat{U}^{*}_{t}(s)}|<2(T-t)\mathbb{V}\mathbb{L}\varepsilon$ for all $t,s$; 2. 2.

In the infinite horizon, $|{U^{*}(s)}-{\hat{U}^{*}(s)}|<2\mathbb{V}\mathbb{L}\varepsilon/(1-\gamma)$ for all $s$, where $U^{*}$ is the optimal time independent strategy.

The proof of 1 is in Appendix B.4, and the proof of 2 in Appendix C.1.

4.2 Learning the Underlying Distribution from Samples

As discussed above, we show here how to compute a $\hat{Q}$ from samples of $Q$, such that $|Q-\hat{Q}|$ is small with high probability. In particular we present a sampling procedure which respects the rules of the pricing server mechanism. When a job arrives, we only learn its length, and only if it agrees to be scheduled. Thus, we are not given full samples of $Q$, complicating the learning procedure. Thanks to the previous section, we know that a policy which is optimal with respect to $\hat{Q}$ will be close-to-optimal with respect to $Q$.

We remark, however, that the power of the results of the previous section is not exhausted by this application: one may apply directly the robustness results to specific problems in which the $\hat{Q}$ is subject to (small) noise or an approximate distribution is already known from other sources.

Let $\mathcal{X}=\{(L_{1},V_{1},D_{1}),\,\dotsc,$ $\,(L_{n},V_{n},D_{n}),\}$ be an i.i.d. sample of $n$ jobs from the underlying distribution $Q$. Note that the expectation of an indicator is the probability of the indicated event. Fix a length $\ell$, a state $s$, and a value $v$. As a consequence of Höffding’s inequality, with probability $1-\delta$,

[TABLE]

Sampling Procedure.

We wish to estimate the value $\mathbb{P}[L=\ell,V\geq v,D\geq s]$ for all choices of $\ell$, $v$, and $s$. Fixing $v$ and $s$, we may repeatedly post prices $\pi_{t}(s,\ell)=v$ and declare that the earliest available time is $s$, then record (i) which job accepts to be scheduled, and (ii) the length of each scheduled job. Let $\underline{\varepsilon}>0$ and $n\geq\log(2/\delta)/(2\underline{\varepsilon}^{2})$, then by (10), the sample-average of each value will have error at most $\underline{\varepsilon}$ with probability $1-\delta$, for any one choice of $(\ell,v,s)$.

Repeating this process for all $\leq\mathbb{K}^{2}$ choices of $v\in\mathcal{V}$ and $s\in\SS$ gives us estimates for each. Now, if we want to have the estimate hold over all choices of $\ell,v,s$, it suffices to take the union bound over all $\leq\mathbb{K}^{3}$ values (incl. $\ell$), and scaling accordingly. If we take $n\geq 3\log(2\mathbb{K}/\delta)/(2\underline{\varepsilon}^{2})$ samples for each of the $\leq\mathbb{K}^{2}$ choices of $v$ and $s$, then simultaneously for all $\ell$, $v$, and $s$, the quantity in (10) is at most $\underline{\varepsilon}$. So we have obtained the “$|Q-\hat{Q}|<\underline{\varepsilon}$” condition. It should be noted that, for this sampling procedure, if a job of length $\ell$ is scheduled, we must possibly wait at most $\ell$ times units before taking the next sample to clear the buffer. This blows up the sampling time by a factor of $O(\mathbb{L})$. The following result follows immediately from Lemma 7 and Höffding’s inequality for the right choice of $n$.

Lemma 8**.**

Let $n$, $Q$, and $\hat{Q}$, be as above. In the finite horizon, for all $\varepsilon>0$, if $n\geq 6T\mathbb{K}^{4}\log(2\mathbb{K}/\delta)/\varepsilon^{2}$, we have that with probability $1-\delta$, $|{U^{*}_{t}(s)}-{\hat{U}^{*}_{t}(s)}|<\varepsilon$ for all $t,s$. In the infinite horizon, if $n\geq 6\mathbb{K}^{4}\log(2\mathbb{K}/\delta)/((1-\gamma)\varepsilon^{2})$, we have that with probability $1-\delta$, $|{U^{*}(s)}-{\hat{U}^{*}(s)}|<\varepsilon$ for all $s$.

4.3 Performance of the Computed Policy

We use here the result of the previous sections to analyze the performance of the policy output by Algorithm 2 after the learning procedure. By the estimation of revenue, the best policy in estimated-expectation is near-optimal in expectation. Since revenues from arbitrary policies concentrate, we get near-optimal revenue in hindsight.

Formally, for $\varepsilon>0$, Lemma 8 gives us that if the sample-distribution $\hat{Q}$ is computed on $n\geq 6T\mathbb{K}^{4}\log(2\mathbb{K}/\delta)/\varepsilon^{2}$ samples, then with probability $1-\delta$ over the samples, $|{U^{*}_{t}(s)}-{\hat{U}^{*}_{t}(s)}|\leq\varepsilon$. Note that ${U^{*}_{t=0}(s=0)}$ is exactly the expected cumulative revenue of the optimal policy. For clarity of notation, denote

[TABLE]

We have shown that for sufficient samples, $|\mathsf{ECRev}_{T}(\pi^{*}|Q)-\mathsf{ECRev}_{T}(\pi^{*}|\hat{Q})|<\varepsilon$, with probability $1-\delta$. This observation allows us to then conclude

Theorem 9** (Finite Horizon).**

Let $Q$ be the underlying distribution over jobs. Let $\varepsilon>0$, and $n\geq 24T\mathbb{K}^{4}\log(8\mathbb{K}/\delta)/\varepsilon^{2}$. Then in time $O(T\mathbb{K}^{3}+n\mathbb{L})$, we may compute a policy $\hat{\pi}$ which is monotone in length, and therefore incentive compatible, such that for any policy $\pi$, with probability $(1-\delta)$,

[TABLE]

Furthermore, if the distribution over values $V$ is continuous rather than discrete, we may compute in time $O(T\mathbb{K}^{2}\mathbb{V}/\eta+n\mathbb{L})$ a monotone policy $\hat{\pi}$ such that for any policy $\pi$, with probability $1-\delta$,

[TABLE]

Proof.

We have chosen $n\geq 6T\mathbb{K}^{4}\log(2\mathbb{K}/(\delta/4))/(\varepsilon/2)^{2}$. Let $\pi^{*}$ be the optimal policy for the true distribution $Q$. By Theorem 6, we have $|\mathsf{CmlRev}_{T}(\mathcal{X},\pi)-\mathsf{ECRev}_{T}(\pi|Q)|<\mathbb{V}\sqrt{2\log(8/\delta)(T+1)}$ with probability $1-\delta/4$ for both $\pi$ and $\hat{\pi}$. Furthermore, by Lemma 8, $|\mathsf{ECRev}_{T}(\pi|Q)-\mathsf{ECRev}_{T}(\pi|\hat{Q})|<\varepsilon/2$ with probability $1-\delta/4$, for both $\pi=\hat{\pi}$ and $\pi^{*}$. This is because from the point of view of $\hat{\pi}$, $\hat{Q}$ is the true distribution, and $Q$ is the estimate. Taking the union bound over all four events above, and recalling that $\hat{\pi}$ maximizes $\mathsf{ECRev}_{T}(\pi|\hat{Q})$, and $\pi^{*}$ maximizes $\mathsf{ECRev}_{T}(\pi|Q)$, we get the following with probability $1-\delta$:

[TABLE]

as desired.

When $V$ is continuously distributed, choose prices which are multiples of $\eta$ between 0 and $\mathbb{V}$, as is outlined in Appendix C.2. ∎

For what concerns the $\gamma$-discounted infinite horizon case, we have the following

Theorem 10** (Infinite Horizon, Discounted).**

Let $Q$ be the underlying distribution over jobs. Let $\varepsilon>0$, and $n\geq 24\mathbb{K}^{4}\tfrac{\log(8\mathbb{K}/\delta)}{\varepsilon^{2}(1-\gamma)}$. Then we may compute a policy $\hat{\pi}$ in time $O\left(\tfrac{\mathbb{K}^{3}}{1-\gamma}\ln\left(\tfrac{\mathbb{V}}{\varepsilon(1-\gamma)}\right)+n\mathbb{L}\right)$, which is monotone, and thus incentive compatible, such that for any policy $\pi$, with probability $(1-\delta)$,

[TABLE]

Furthermore, if the distribution over values $V$ is continuous rather than discrete, we may compute in time $O\left(\tfrac{\mathbb{K}^{2}\mathbb{V}/\eta}{1-\gamma}\ln\left(\tfrac{\mathbb{V}}{\varepsilon(1-\gamma)}\right)+n\mathbb{L}\right)$ a monotone policy $\hat{\pi}$ such that for any $\pi$, with probability $1-\delta$,

[TABLE]

As above, this policy $\hat{\pi}$ is computed by learning $\hat{Q}$ from $n$ samples as in Section 4.2, then running the modified Algorithm 2 for the estimated distribution as in Appendix C.1. In case $V$ is continuously distributed, we restrict ourselves to prices which are multiples of $\eta$ between 0 and $\mathbb{V}$. The details of the proof are in Appendix C.

We recall that all these results need the distribution assumption from Section 3.3.

5 Conclusions and Future Work

In summary, we propose to price time on a server by first learning the distribution over jobs from samples, then computing the Bayes-optimal policy from the estimated distribution. Our learning algorithm is simple: we sample the distribution through the observation of $n$ jobs at artificially fixed prices and server-states, and learn the job parameters depending on whether they accept to be scheduled. Using these observations, we build an observed distribution $\hat{Q}$. We then run Algorithm 2 with $\hat{Q}$ and compute an optimal policy $\hat{\pi}$ for $\hat{Q}$. We are guaranteed that the policy prices monotonically (due to Lemma 3), and therefore it is incentive compatible, which implies the correctness of the estimated revenue.

Future Work.

There are many natural extensions to this work. For example, one could consider a multi-server setting, settings where jobs can request to be scheduled later than the earliest available time, or settings where jobs need various quantities of differing resources, such as memory and computation time.

Appendix A Log-Concave Distributions

In Section 3.3, we sought to show that if the value of a random job has a log-concave distribution, then the optimal policy will be monotone. We present here a discussion of log-concavity, both for continuous and discrete random variables, and give the proof of the monotonicity of the prices.

Formally, a function $f:\mathbb{R}\to\mathbb{R}$ is log-concave if for any $x$ and $y$, and for any $0\leq\theta\leq 1$, $\lg f(\theta x+(1-\theta)y)\geq\theta\lg f(x)+(1-\theta)\lg f(y)$. Equivalently, $f(\theta x+(1-\theta)y)\geq f(x)^{\theta}f(y)^{1-\theta}$. For a discretely supported $f:\mathbb{Z}\to\mathbb{R}$, we replace this condition with $f(x)^{2}\geq f(x-1)f(x+1)$, emulating the continuous definition with $\theta=\tfrac{1}{2}$. We further require that the support of $f$ be “connected”.

Definition 11**.**

A continuous random variable $X$ with density function $f$ is said to be log-concave if $f$ is log-concave. A discrete random variable $Y$ with probability mass function $p$ is said to be log-concave if $p$ is discretely log-concave.

A well-known fact is that log-concave random variables also have log-concave cumulative density/mass functions. We present here a quick proof of this fact, for completeness.

Claim 12**.**

If $X$ is a log-concave continuous r.v., then $\mathbb{P}[X\leq x]$, and ${\mathbb{P}[X\geq x]}$ are log-concave functions of $x$. If $Y$ is a log-concave discrete r.v. supported on $\mathbb{N}$, then $\mathbb{P}[Y\leq y]$ and $\mathbb{P}[Y\geq y]$ are discretely log-concave functions of $y$.

Proof.

The continuous case is well-documented in the literature. See for example [24]. For the discrete case, observe first that since a mass function is non-negative, and we have assumed contiguous support, the function must be single-peaked, i.e. quasi-concave, as any local minimum would contradict the definition. Furthermore, the definition of log-concavity is equivalent to $\tfrac{p_{y}}{p_{y-1}}\geq\tfrac{p_{y+1}}{p_{y}}$. Repeatedly applying this, and rearranging, we get

[TABLE]

It remains to show that $P(y):=\sum_{-\infty}^{y}p_{k}$ is log-concave. We have

[TABLE]

as desired. The same technique applies for the upper-sum. ∎

This will allow us to then conclude:

(Lemma 2, p.2)

Let, $V_{\ell}^{s}$ denote the marginal r.v. $V$ conditioned on $L=\ell$ and $D\geq s$. Let $Z$ be a continuously-supported random variable, and $\gamma_{1}^{s}\leq\gamma_{2}^{s}\leq\dotsm\in\mathbb{R}$. If $V_{\ell}^{s}$ is distributed like $\gamma_{\ell}^{s}\cdot Z$, $\left\lfloor\gamma_{\ell}^{s}\cdot Z\right\rfloor$, $Z+\gamma_{\ell}^{s}$, or $\left\lfloor Z+\gamma_{\ell}^{s}\right\rfloor$, then Assumption 1 is satisfied if $Z$ is log-concave, or if the $\gamma$’s are independent of $\ell$.

Proof.

First, observe that

[TABLE]

and since we are taking ratios for $s$ fixed, we can replace the joint cumulatives on $V$ and $D$ in the assumption, with the marginals on just $V$.

Now, if the $\gamma$’s are independent of $\ell$, then the ratio remains unchanged as $\ell$ changes, satisfying assumption 1. Otherwise, we begin by analyzing the distributions given by $\gamma_{\ell}^{s}Z$ and $Z+\gamma_{\ell}^{s}$. Let $\bar{F}(x):=\mathbb{P}[Z\geq x]$, noting that $\mathbb{P}[V_{\ell}^{s}\geq{\mu}]=\bar{F}({\mu}/\gamma_{\ell}^{s})$ and $\bar{F}({\mu}-\gamma_{\ell}^{s})$, for the two cases, respectively. Note that we wish to show $\mathbb{P}[V_{\ell}^{s}\geq{\mu}^{\prime}]/\mathbb{P}[V_{\ell}^{s}\geq{\mu}]$ is increasing, which is equivalent to $\log(\mathbb{P}[V_{\ell}^{s}\geq{\mu}^{\prime}])-\log(\mathbb{P}[V_{\ell}^{s}\geq{\mu}])$ increasing.

For $V_{\ell}^{s}\sim Z+\gamma_{\ell}^{s}$, observe that for $x^{\prime}>x$ and $\gamma^{\prime}>\gamma$, we have

[TABLE]

since $\log\bar{F}$ is a non-increasing and concave function, by assumption. Also

[TABLE]

where the first inequality is the same as the previous equation, as the second is by monotonicity. Thus we have done the continuous case.

For $V_{\ell}^{s}\sim\left\lfloor Z+\gamma_{\ell}^{s}\right\rfloor$, we note that $\left\lfloor Z+\gamma\right\rfloor\geq x$ if $Z+\gamma\geq\left\lceil x\right\rceil$. So the probability is $\bar{F}(\left\lceil x\right\rceil-\gamma)$. Similarly, for $V_{\ell}^{s}\sim\left\lfloor\gamma_{\ell}^{s}\cdot Z\right\rfloor$, $\mathbb{P}\left\lfloor\gamma Z\right\rfloor\geq x$ is $\bar{F}(\left\lceil x\right\rceil/\gamma)$. Thus, if we assume that $x$ and $x^{\prime}$ are integers, the calculations above go through, as desired. ∎

We present a final fact that justifies the use of $\left\lfloor Z\right\rfloor$-type random variables:

Lemma 13**.**

If $Y$ is a discrete log-concave random variable, then there exists a continuous log-concave $Z$ such that $Y\sim\left\lfloor Z\right\rfloor$.

Proof.

Let $P:\mathbb{Z}\to[0,1]$ be the right-hand cumulative mass function for $Y$. Then, it suffices to have $\mathbb{P}[Z\geq n]=P(n)$ for all integers $n$. Let $\phi:\mathbb{R}\to\mathbb{R}$ be the piecewise-linear function such that $\phi(-\infty)\to 0$, $\phi(\infty)\to-\infty$, and $\phi(n)=\log(P(n))$ for all $n$. Since $\log(P)$ is a discretely concave and non-increasing function, $\phi$ must be concave and non-increasing. We can then set $Z$ to be the random variable whose density is given by $-\tfrac{\mathrm{d}}{\mathrm{d}x}\exp(\phi(x))$. ∎

Appendix B Detailed Proofs

We present in this section the detailed proofs of the lemmas and theorems from the text. B.1 gives the pseudocode for the dynamic programs that compute the optimal pricing policies, outlined in Section 3, B.2 gives the proofs for the monotonicity of the pricing policies, along with the discussion on log-concave random variables from Appendix A,  B.3 gives the concentration bounds from the last part of Section 3, and  B.4 deals with the proof of Section 4 .

B.1 MDP Algorithms and Correctness

(Lemma 1, p.1)

For any fixed pricing policy $\pi:[T]\times\SS\times\mathcal{L}\to\mathbb{R}$,

[TABLE]

*where the ${U^{\pi}_{t}(\cdot)}$’s are as in (2), and the ${u^{\pi}_{t}(\cdot,\cdot)}$’s are from the modified MDP. *

Proof.

The statement is true for $t=T$ since in that case everything is zero. Suppose $\operatorname{\mathbb{E}}\limits_{L^{\prime}}\left[{u^{\pi}_{t+1}(s,L^{\prime})}\right]={U^{\pi}_{t+1}(s)}$ for all $s$. For the fixed policy $\pi$, we define $\bm{p}^{\ell}_{t,s}:=\mathbb{P}[V\geq\pi_{t}(s,\ell),D\geq s|L=\ell]$. Then,

[TABLE]

∎

B.2 Monotonicity of Prices

These proofs are given in parallel with the discussion in Appendix A.

(Lemma 3, p.3)

*Let ${U^{*}_{t}(s)}$ be the expected future revenue earned starting at time $t$ in state $s$, for the optimal policy computed by Algorithm 2. Then the function $s\mapsto{U^{*}_{t}(s)}$ is monotone non-increasing in $s$ for any $t$ fixed. *

Proof.

The proof is by induction on the time, decreasing. At time $t=T$, there is no future revenue and ${U^{*}_{T}(s)}=0$, so the inductive claim follows trivially. Suppose, now, that the inductive claim holds at time $t+1$. It suffices to show that this holds for each ${u^{*}_{t}(s,\ell)}$, since ${U^{*}_{t}(s)}$ is simply their expectation. Let $\pi^{*}_{t}$ be the optimal pricing policy computed for the time $t$ by the Algorithm 2. Since the function ${\mu}\mapsto\mathbb{P}[V\geq{\mu}\ \text{and}\ \mathcal{E}]$, for any event $\mathcal{E}$, is left-continuous in the variable ${\mu}$, we may define, for every $\ell\in\mathcal{L}$ and $s\in\SS,$

[TABLE]

We must have ${\mu}^{\prime}\geq\pi^{*}_{t}(s+1,\ell)$, as $\mu=\pi^{*}_{t}(s+1,\ell)$ is in the set. Now, letting $\bm{p}:=\mathbb{P}[V\geq\pi^{*}_{t}(s+1,\ell),D\geq s+1|L=\ell]$, we have

[TABLE]

where $(x)_{+}:=\max\{x,0\}$. The first inequality holds by the induction hypothesis, the second is by definition of $(\cdot)_{+}$, the third by the definition of ${\mu}_{s}^{\prime}$, and in the last, from the fact that ${\mu}_{s}^{\prime}$ is a (possibly) suboptimal pricing policy for the state $s$ at time $t$. Note that this last inequality requires that the 0 value be feasible in the max, which it is, by setting $\mu^{\prime}$ arbitrarily large. ∎

(Lemma 4, p.4)

If the distribution on job parameters satisfies assumption 1, then for all $\ell,s,t$, we have $\pi^{*}_{t}(s,\ell)\leq\pi^{*}_{t}(s,\ell+1)$.

Proof.

Let $\bm{p}^{\ell}_{s}({\mu}):=\mathbb{P}[V\geq{\mu},D\geq s|L=\ell]$, fix $s$, $t$, and $\ell$, and let ${\mu}_{0}$ be equal to the optimal price $\pi^{*}_{t}(s,\ell)$. Observe that ${\mu}_{0}$ maximizes the expression

[TABLE]

For simplicity, let $\Delta_{\ell}:={U^{*}_{t+1}(s+\ell-1)}-{U^{*}_{t+1}(s-1)}$, and so for any ${\mu}\neq{\mu}_{0}$,

[TABLE]

Note that, as discussed in the proof of the previous lemma, $\mu_{0}+\Delta_{\ell}\geq 0$, as otherwise it would be beneficial to set $\pi_{t}^{*}(s,\ell)\leftarrow\infty$. The above inequality is then equivalent to

[TABLE]

We wish to show that, if ${\mu}\leq{\mu}_{0}$, then as $\ell$ increases, the above inequality still holds. This would imply that the price ${\mu}_{0}=:\pi_{9}^{*}(s,\ell)$ gives better return than ${\mu}$ for jobs of length $\ell+1$, implying that the optimal price must be at least $\pi_{t}^{*}(s,\ell)$, which is our desired goal.

Now, by assumption 1, the left-hand-side is non-decreasing in $\ell$, so it remains to show that the right-hand-side is non-increasing in $\ell$. The only changing term is $\Delta_{\ell}$, which by Lemma 3, is non-increasing in $\ell$. Since it is in the denominator of a subtracted, non-negative term, we have our desired result. ∎

B.3 Concentration Bounds on Revenue for Online Scheduling

(Theorem 6, p. 6)

Let $\mathcal{X}$ be a finite sequence of jobs sampled $i.i.d.$ from an underlying distribution $Q$, and let $\pi$ be any monotone policy. Then, with probability $1-\delta$,

[TABLE]

*in the finite horizon, and in the infinite-horizon-discounted,

[TABLE]

In particular these results hold true for the (approximately) optimal pricing strategy computed in the previous part of the section.

Proof.

For the finite horizon, we apply Azuma’s inequality to the martingale $R_{t}^{\pi}$. We being by showing the bounded-differences property. Note that we do not require truthful behaviour from the jobs, since taking strategic behaviour into account for a non-monotone policy is equivalent to modifying the distribution over the jobs, and making the distribution state-dependent, by increasing the length of those jobs who would rather buy a longer interval. Thus,

[TABLE]

where the last inequality follows from properties of conditional expectation. With this property, we can apply Azuma’s, and get

[TABLE]

For the infinite-horizon-discounted, we observe that equation (7) becomes

[TABLE]

and thus we get that $|R_{t}^{\pi}-R_{t-1}^{\pi}|\leq\gamma^{t}\mathbb{V}$. Therefore with probability $1-\delta$,

[TABLE]

Thus, taking the limit as $T\to\infty$, we get that with probability $1-\delta$,

[TABLE]

∎

B.4 Robustness of the pricing strategy

(Lemma 7, p. 7)

Let $Q$, and $\hat{Q}$ such that $|Q-\hat{Q}|<\varepsilon$. In the finite horizon, $|{U^{*}_{t}(s)}-{\hat{U}^{*}_{t}(s)}|<2(T-t)\mathbb{V}\mathbb{L}\varepsilon$ for all $t,s$.

Proof.

Let $\pi^{*}$ be the policy computed by Algorithm 2 with access to $Q$. As in Section 3, we denote $\bm{p}^{\ell}_{t,s}:=\mathbb{P}[V\geq\pi^{*}_{t}(s,\ell),D\geq s|L=\ell]$, and ${\bm{P}}(\ell):=\mathbb{P}_{\mathcal{X}}[L=\ell]$. In an abuse of notation, denote $\hat{\bm{p}}^{\ell}_{t,s}$ and $\hat{\bm{P}}(\ell)$ the estimated values of $\bm{p}^{\ell}_{t,s}$ and ${\bm{P}}(\ell)$, respectively. We cannot estimate $\bm{p}^{\ell}_{t,s}$ directly with good error bounds, but we will only need the values $\hat{\bm{P}}(\ell)\hat{\bm{p}}^{\ell}_{t,s}$ and $\hat{\bm{P}}(\ell)(1-\hat{\bm{p}}^{\ell}_{t,s})$. Now, substituting these estimates into (9), we get:

[TABLE]

To simplify this expression, we begin by showing a simple claim: Let $x$, $y$, $\hat{x}$, $\hat{y}\in\mathbb{R}$, and let $\lambda,\hat{\lambda}\in[0,1]$, such that $|x-\hat{x}|<\delta$, $|y-\hat{y}|<\delta$, and $|\lambda-\hat{\lambda}|<\varepsilon$. Then

[TABLE]

Now, replacing $x$ and $y$ with \big{(}\pi^{*}_{t}(s,\ell)+{U^{*}_{t+1}(s+\ell-1)}\big{)} and ${U^{*}_{t+1}(s-1)}$, respectively, and replacing $\lambda$ with $\bm{P}(\ell)\bm{p}^{\ell}_{t,s}$, we have

[TABLE]

However, the argument of the supremum in left-hand term in the summand must be at most $\mathbb{V}$, since if ${U^{*}_{t+1}(\sigma+\ell-1)}\leq{U^{*}_{t+1}(s-1)}$, it is best to $\pi^{*}_{t}(\sigma)=\infty$, which makes $\bm{p}^{\ell}_{t,s}=0$, putting all the weight on ${U^{*}_{t+1}(s-1)}$. Furthermore, we have shown in Lemma 3 that ${U^{*}_{t+1}(s+\ell-1)}\leq{U^{*}_{t+1}(s-1)}$. Thus, we get

[TABLE]

Inductively applying this gives $\left|{U^{*}_{t}(s)}-{\hat{U}^{*}_{t}(s)}\right|\leq 2(T-t)\mathbb{L}\mathbb{V}\underline{\varepsilon}$ as desired. ∎

Appendix C Extensions

In this section, we extend the finite-horizon results to compute the optimal policies in the infinite-horizon-discounted setting, and also to argue that the optimal policy may be computed within some error when the distribution over values is continuous, rather than discrete.

These results are needed to show the full statements of Theorems 5–10.

C.1 Infinite Discounted Horizon

Recall, in this infinite horizon discounted setting, we seek to maximize the $\gamma$-discounted future revenue,

[TABLE]

over the choice of $\pi:\mathbb{N}\times\SS\times\mathcal{L}\to\mathbb{R}$. Algorithm 2 does not allow us to immediately compute a solution for the infinite discounted horizon case. However we can exploit the discounting factor on the revenues to obtain an approximation of the infinite optimum: it suffices to consider the truncated problem up to a certain sufficiently large $T$ and solve it optimally using the algorithm presented above. In fact we have the following Lemma.

Lemma 14**.**

For any $\varepsilon>0$ and $T\geq\log_{\gamma}\left(\varepsilon(1-\gamma)/\mathbb{V}\right)$, let $\pi$ be the pricing policy computed by the finite-horizon algorithm up to time $T$. Let $\bar{\pi}$ be the time-independent pricing policy such that $\bar{\pi}(\cdot,\cdot):=\pi_{0}(\cdot,\cdot)$. Then the expected performance of the optimal policy in the infinite horizon is within an additive $\varepsilon$ of expected performance of $\bar{\pi}$.

Proof.

Note that in order to compute policy $\pi$ it is necessary to add the discount factor to Algorithm 2, and to all of the proofs of previous sections. One can verify that all proofs go through. Let $\pi^{*}$ be the Bayes-optimal infinite-horizon strategy — which is known to be time-independent — and let $\pi$ be as in the statement (where we set $\pi_{t}(s,\ell)=\infty$ for all $t>T$.) Then, in expectation over times [math] through $T$, pricing as $\pi$ yields greater revenue than following $\pi^{*}$. Conversely, in expectation over all time, pricing as $\pi^{*}$ yields greater revenue than $\pi$. However, after time $T$, the maximum possible revenue due to any policy is

[TABLE]

And so the difference in revenue due to following $\pi$ or $\pi^{*}$ is at most $\varepsilon$, since $T$ is sufficiently large.

It remains to show that $\bar{\pi}$ performs better than $\pi$ overall. Let $\pi^{i}$ be the policy which agrees with $\pi_{0}$ for all $t\leq i$, then equals $\pi_{t-i}$ for $t>i$. Observe that, $\pi^{1}$ is optimal in expectation over the interval $[1,T+1]$, and is equivalent to $\pi=\pi^{0}$ for the first step. Therefore, $\pi^{1}$ performs better than $\pi$. Similarly, we can argue $\pi^{i+1}$ performs better than $\pi^{i}$ over the interval $[i,T+i]$ and equally before, hence performs better overall.

Thus, we have a sequence of policies $\pi=\pi^{0},\,\pi^{1},\,\pi^{2},\,\dotsc$ converging to $\bar{\pi}$, and whose expected revenue is monotone non-decreasing along the sequence. Therefore, the expected revenue due to $\bar{\pi}$ is greater than that of $\pi$, which is an $\varepsilon$ additive-approximation to the optimal policy. ∎

The approach above is analogous to the classical value iteration technique [23].

(Lemma 7, p. 7)

Let $Q$, and $\hat{Q}$ such that $|Q-\hat{Q}|<\varepsilon$. In the infinite horizon, $|{U^{*}(s)}-{\hat{U}^{*}(s)}|<2\mathbb{V}\mathbb{L}\varepsilon/(1-\gamma)$ for all $s$.

Proof.

As in the proof of Lemma 14, if $T$ is sufficiently large, we may analyze the first $T$ time steps as a finite-horizon problem, and the remaining revenue will be negligibly small. Now, the calculation above can be reproduced with discount terms, to show

[TABLE]

Then, inductively applying this and taking $T\to\infty$, we have $|{U^{*}_{0}(s)}-{\hat{U}^{*}_{0}(s)}|\leq 2\mathbb{V}\mathbb{L}\underline{\varepsilon}/(1-\gamma)$. ∎

These results are used to prove the infinite-horizon versions of the various results throughout the paper, specifically the Theorems 5–6, and 10.

C.2 Approximation Algorithm for Continuously Supported Values

Note that the algorithms above assume that the value of the jobs ($V$) is discretely supported, and the running time depends on $|\mathcal{V}|$. In this section, we analyze the error incurred by discretizing the space of possible values, and then computing the optimal policy.

Let $\eta>0$ be some desired small grid size, and suppose we only allow ourselves to set prices which are multiples of $\eta$. We claim that this incurs a small loss on the total revenue.

Define, as in the previous subsections, $\bm{p}^{\ell}_{s}({\mu}):=\mathbb{P}[V\geq{\mu},D\geq s|L=\ell]$. Further, define as previously ${U^{*}_{t}(s)}=\operatorname{\mathbb{E}}\limits_{L}\left[{u^{*}_{t}(s,L)}\right]$, and

[TABLE]

Define ${U^{*}_{t,\eta}(s)}$ and ${u^{*}_{t,\eta}(s,\ell)}$ similarly, restricting the maximum to choosing ${\mu}$ from multiples of $\eta$.

Lemma 15**.**

Let ${U^{*}_{t}(\cdot)}$ and ${U^{*}_{t,\eta}(\cdot)}$ be defined as above, then $|{U^{*}_{t}(s)}-{U^{*}_{t,\eta}(s)}|\leq(T-t)\eta\ \forall s,t$.

Proof.

We will show this by induction on the value of $t$, decreasing. Assume that $|{U^{*}_{t}(s)}-{U^{*}_{t,\eta}(s)}|<\Delta_{t}$ for all $t$, $s$, and set $\Delta_{T+1}=0$. We wish to inductively bound the value of $\Delta_{t}$. Now,

[TABLE]

Now, let ${\mu}^{*}$ be the optimizer of this right hand side over $\mathbb{R}$ (where the value would attain ${u^{*}_{t}(s,\ell)}$), and $\hat{\mu}$ be ${\mu}^{*}$ rounded down to the nearest multiple of $\eta$. Then, since $\bm{p}^{\ell}_{s}(\cdot)$ is non-increasing,

[TABLE]

Thus combining both equations, we get

[TABLE]

From which we conclude, by averaging over $\ell$, that $\Delta_{t}\leq(T-t)\eta$, as desired. ∎

Corollary 16**.**

Let ${U^{*}(\cdot)}$ and ${U^{*}_{\infty,\eta}(\cdot)}$ be defined as above, but for the infinite horizon discounted, then $|{U^{*}(s)}-{U^{*}_{\infty,\eta}(s)}|\leq\eta/(1-\gamma)\ \forall s$.

Proof.

As shown in the previous subsection, it suffices to perform the analysis in the finite horizon, while taking the discount factor into account, then take the limit as $T\to\infty$. The same calculations as above gives

[TABLE]

Summing the $\Delta$’s and taking $T\to\infty$, we get ${u^{*}_{\infty,\eta}(s,\ell)}\geq{u^{*}(s,\ell)}-\eta/(1-\gamma)$ as desired. ∎