Competitive Algorithms for Online Budget-Constrained Continuous DR-Submodular Problems

TL;DR

This paper introduces a primal-dual algorithm for online optimization of DR-submodular functions under budget constraints, achieving a competitive ratio that matches the best known bounds in special cases.

Contribution

It provides the first competitive ratio bound for online monotone DR-submodular maximization with linear packing constraints, extending prior results.

Findings

First bound on competitive ratio for this class of problems

Matches tight bounds in the linear case

Applicable to online monotone DR-submodular maximization

Abstract

In this paper, we study a certain class of online optimization problems, where the goal is to maximize a function that is not necessarily concave and satisfies the Diminishing Returns (DR) property under budget constraints. We analyze a primal-dual algorithm, called the Generalized Sequential algorithm, and we obtain the first bound on the competitive ratio of online monotone DR-submodular function maximization subject to linear packing constraints which matches the known tight bound in the special case of linear objective function.

Tables

Table 1. (a) n = 1 , m = 100 , K = 50 formulae-sequence 𝑛 1 formulae-sequence 𝑚 100 𝐾 50 n=1,m=100,K=50

Quantity	Value ($\%$)
Competitive Ratio	64.33
Budget Usage	74.95

Table 2. (a) n = 1 , m = 100 , K = 50 formulae-sequence 𝑛 1 formulae-sequence 𝑚 100 𝐾 50 n=1,m=100,K=50

Quantity	Value ($\%$)
Competitive Ratio	64.33
Budget Usage	74.95

Table 3. (b) n = 5 , m = 100 , K = 50 formulae-sequence 𝑛 5 formulae-sequence 𝑚 100 𝐾 50 n=5,m=100,K=50

Quantity	Value ($\%$)
Competitive Ratio	58.27
Budget $1$ Usage	65.68
Budget $2$ Usage	58.06
Budget $3$ Usage	66.83
Budget $4$ Usage	65.11
Budget $5$ Usage	74.75

Equations

$$F^{*}(y)=\inf_{x}\big{(}\langle x,y\rangle-F(x)\big{)}$$

$${\sigma }_{\mathcal{P}}(x)=\underset{y\in \mathcal{P}}{\sup }⟨x,y⟩$$

$$x⪰y\Rightarrow \nabla F(x)⪯\nabla F(y)$$

$$F(x+tv)\le F(x)+t⟨\nabla F(x),v⟩$$

$$f(A\cup \{j\})-f(A)\ge f(B\cup \{j\})-f(B)$$

$$F(x)=\sum _{S\subset V}f(S)\prod _{i\in S}{x}_i\prod _{j\notin S}(1-x_j)={\mathbb{E}}_{S\sim x}[f(S)]$$

$$\begin{array}[]{ll}\mbox{maximize}&\sum_{i=1}^{n}H_{i}(\hat{x}_{i})\\ \mbox{subject to}&x_{t}\in F_{t}\subseteq\mathbb{R}_{+}^{n}~{}\forall t\in[m]\\ &\hat{c}_{i}^{T}\hat{x}_{i}\leq 1~{}\forall i\in[n]\end{array}$$

$$\begin{array}[]{ll}\mbox{maximize}&\sum_{i=1}^{n}\big{(}H_{i}(\hat{x}_{i})+G_{i}(\hat{c}_{i}^{T}\hat{x}_{i})\big{)}\\ \mbox{subject to}&x_{t}\in F_{t}\subseteq\mathbb{R}^{n}~{}\forall t\in[m]\\ \end{array}$$

$$G_i(u)=\{\begin{array}{ll}0& \text{if }0\le u\le 1\\ -\infty & \text{if }u>1\end{array}$$

$$\begin{array}[]{ll}\mbox{minimize}&\sum_{t=1}^{m}\sigma_{F_{t}}\big{(}\begin{bmatrix}y_{1,t}-z_{1}c_{1,t}\\ \vdots\\ y_{n,t}-z_{n}c_{n,t}\end{bmatrix}\big{)}-\sum_{i=1}^{n}H_{i}^{*}(\hat{y}_{i})+\sum_{i=1}^{n}z_{i}\\ \mbox{subject to}&z_{i}\geq 0~{}\forall i\in[n]\\ \end{array}$$

$$x_t^{\ast }$$

$${\hat{y}}_i^{\ast }$$

$$z_i^{\ast }$$

$${\omega }_{i,t}(k)$$

$$[d_t(k){]}_i$$

$$\mathrm{ALG}$$

$${\mathrm{P}}_{\mathrm{gseq}}$$

$$H_i(x+u)-H_i(x)\ge ⟨u,\nabla H_i(x)⟩-\frac{L}{2}\Vert u{\Vert }^2$$

$$G_i(x+v)-G_i(x)\ge v{G}_i^{\prime }(x)-\frac{L}{2}{v}^2$$

$${\alpha }_{H_i}$$

$$=\underset{u:{\hat{c}}_i^{T}u\le 1}{\sup }\{\beta |⟨\nabla H_i(u),u⟩\ge (1+\beta )H_i(u)\}$$

$$=\underset{u:{\hat{c}}_i^{T}u\le 1}{\inf }\frac{⟨\nabla H_i(u),u⟩}{H_i(u)}-1$$

$${\kappa }_f=1-\underset{j:f(j)\ne 0}{\min }\frac{f(j|(V\setminus j))}{f(j)}=1-\underset{S\subset V\setminus \{j\},f(j)\ne 0}{\min }\frac{f(j|S)}{f(j)}$$

$${\alpha }_F\ge -{\kappa }_f$$

$$G_{1}(u)=\begin{cases}-L_{1}u&\quad\text{if }0\leq u<\frac{1}{\ln\big{(}\frac{U_{1}e}{L_{1}}\big{)}}\\ -\frac{1}{\ln\big{(}\frac{U_{1}e}{L_{1}}\big{)}}\frac{L_{1}}{e}(\frac{U_{1}e}{L_{1}})^{u}&\quad\text{if }u\geq\frac{1}{\ln\big{(}\frac{U_{1}e}{L_{1}}\big{)}}\\ \end{cases}$$

$$G_i(u)$$

$$U_i$$

$$L_i$$

$$\frac{\rm ALG}{\rm OPT}\geq\frac{\rm ALG}{\rm D^{*}}\geq\big{(}{\max_{i\in[n]}\big{\{}-\alpha_{H_{i}}+\ln(1+\frac{U_{i}(e-1)}{L_{i}})\frac{e}{e-1}\big{\}}}\big{)}^{-1}$$

$$G_{i}(u)=\frac{L_{i}(1+\epsilon)}{(e-1)\ln(1+\frac{U_{i}(e-1)}{L_{i}})}\big{(}1-(1+\frac{U_{i}(e-1)}{L_{i}})^{\frac{u}{1+\epsilon}}\big{)}+\frac{L_{i}}{e-1}u$$

$$\frac{\mathrm{ALG}}{\mathrm{OPT}}\ge \frac{\mathrm{ALG}}{{\mathrm{D}}^{\ast }}\ge \frac{1}{1-{\alpha }_{H_1}+\ln (\frac{U_1}{L_1})}$$

$$G_{1}(u)=\begin{cases}-L_{1}u&\quad\text{if }0\leq u<\frac{1+\epsilon}{\ln\big{(}\frac{U_{1}e}{L_{1}}\big{)}}\\ -\frac{1+\epsilon}{\ln\big{(}\frac{U_{1}e}{L_{1}}\big{)}}\frac{L_{1}}{e}(\frac{U_{1}e}{L_{1}})^{\frac{u}{1+\epsilon}}&\quad\text{if }u\geq\frac{1+\epsilon}{\ln\big{(}\frac{U_{1}e}{L_{1}}\big{)}}\\ \end{cases}$$

$$\begin{array}[]{ll}dy/dt&=v_{\rm max}(y)\\ v_{\rm max}(y)&=\operatorname*{arg\,max}_{v\in\mathcal{P}}\langle v,\nabla F(y)\rangle\end{array}$$

$$\begin{array}[]{ll}\mbox{maximize}&f(V^{\prime})\\ \mbox{subject to}&\sum_{v\in V^{\prime}}c(v)\leq b\\ \end{array}$$

Full text

1 Introduction

Online optimization covers a large number of problems including online resource allocation, online bipartite matching [1], the “Adwords” problem [2, 3], online submodular welfare maximization [4], online linear programming [5] and online concave packing problem [6, 7]. One type of algorithms proposed for solving such problems are primal-dual algorithms where the dual variable is updated at each step and is used to get the update rule for the primal variable [8].

Depending on how much information about the online input is available in advance to the algorithm, online problems have been categorized into adversarial (worst-case) (e.g., in [2]) and stochastic input models (e.g., in [9]) and we consider the former in this paper. In the adversarial model, it is assumed that the algorithm has no knowledge of the online input. Performance of online algorithms is measured by their competitive ratio defined as the ratio of the value of the objective function at the output of the algorithm to the maximum objective value attained offline. In the worst-case model, one is interested in deriving lower bounds on the competitive ratio of the algorithm that holds for all arbitrary online inputs.

In this paper, we discuss a certain class of online optimization problems where the objective function is assumed to satisfy the Diminishing Returns (DR) property under linear packing constraints. We introduce a greedy primal-dual algorithm, called the Generalized Sequential algorithm and we analyze its performance theoretically and numerically under the worst-case input model. Specifically, we make the following contributions:

• •

We introduce the online monotone continuous DR-submodular function maximization subject to linear packing constraints, and specify various online discrete submodular problems whose continuous generalization could be cast in this framework, for example the generalized continuous version of online submodular welfare maximization [4] and online knapsack constrained monotone submodular function maximization [10] are well-known cases.

• •

We introduce the Generalized Sequential algorithm for this class of problems. Denoting the number of linear packing constraints by $n$, we consider the following two cases and we derive competitive ratio bounds for each case:

• –

$n=1$: In this case, our problem is the generalization of online knapsack constrained monotone submodular function maximization [10] to the continous setting. For this online problem, we obtain a competitive ratio of $\frac{1}{1-\alpha+\ln(\frac{U}{L})}$ where $L$ and $U$ are lower and upper bounds on the value-to-weight ratio of the items respectively and $\alpha$ captures the curvature of the DR-submodular utility function.

• –

$n>1$: In this case, our problem generalizes the Adwords problem [2] and the online linear programming problem [5] by allowing the utility function to be DR-submodular rather than linear. For this setting, we obtain the first competitive ratio bound which is optimal in the special cases. Specifically, if the objective function is linear, our problem reduces to the online linear programming and the algorithm achieves the optimal competitive ratio [7, 5]. If in addition to the linearity of the objective function, the linear packing constraint and the objective function are equal, the problem simplifies to the Adwords problem and we obtain the optimal $1-\frac{1}{e}$ competitive ratio [2, 3] (note that since we allow fractional solutions for the Adwords problem, we do not need the small bids assumption to obtain the optimal competitive ratio).

Finally, we present numerical experiments on a class of non-concave DR-submodular utility functions to demonstrate the performance of the Generalized Sequential algorithm.

It is noteworthy that although our framework could be interpreted as the generalization of online budgeted discrete submodular problems to the continuous setting, we do not aim to solve the discrete problem itself. In other words, our goal is to solve a class of online budgeted problems where the objective function is originally continuous and DR-submodular. Therefore, we do not round the fractional output of our proposed algorithm.

1.1 Notation

We will use $[m]$ to denote the set $\{1,2,\dots,m\}$. For a matrix $A\in\mathbb{R}^{n\times m}$, we will denote its $i$-th row by $\hat{a}_{i}^{T}$ for all $i\in[n]$ and its $t$-th column by $a_{t}$ for all $t\in[m]$. Also, $a_{i,t}$ corresponds to the $(i,t)$-th entry of the matrix A. We denote the transpose of a matrix $A$ by $A^{T}$. The inner product of two vectors $x,y\in\mathbb{R}^{m}$ is denoted by either $\langle x,y\rangle$ or $x^{T}y$. Also, for two vectors $x,y\in\mathbb{R}^{m}$, $x\preceq y$ implies that $x_{i}\leq y_{i}~{}\forall i\in[m]$. We use $F^{*}$ to denote the concave conjugate of a function $F:\mathbb{R}^{m}\to\mathbb{R}$ which is defined as follows:

[TABLE]

For a convex set $\mathcal{P}$, the support function of $\mathcal{P}$ is defined in the following:

[TABLE]

2 Diminishing Returns (DR) property

Definition 2.1

A differentiable function $F:K\rightarrow\mathbb{R}$, $K\subset\mathbb{R}_{+}^{m}$, satisfies the Diminishing Returns (DR) property if:

[TABLE]

*In other words, $\nabla F$ is an anti-tone mapping from $\mathbb{R}^{m}$ to $\mathbb{R}^{m}$.

If $F$ is twice differentiable, DR property is equivalent to the Hessian matrix being element-wise non-positive. Note that for $m=1$, the DR property is equivalent to concavity. However, for $m>1$, concavity implies negative semi-definiteness of the Hessian matrix which is not equivalent to the Hessian matrix being element-wise non-positive.*

A similar property is introduced in [11] and [12] as well and functions satisfying this property are called “smooth submodular” and “DR-submodular” there respectively. Additionally, [7] defined the DR property for concave functions with respect to a partial ordering induced by a cone and showed that by taking the cone to be $\mathbb{R}_{+}^{m}$, Definition 2.1 is recovered and if the cone of positive semi-definite matrices is considered, the DR property generalizes to matrix ordering as well [13]. [12] showed that DR-submodular functions are concave along any non-negative direction, and any non-positive direction. In other words, for a DR-submodular function $F$, if $t\geq 0$ and $v\in\mathbb{R}^{m}$ satisfies $v\succeq 0$ or $v\preceq 0$, we have:

[TABLE]

2.1 Examples of continuous non-concave DR-submodular functions

Multilinear extension of discrete submodular functions. [14] A discrete function $f:\{0,1\}^{V}\rightarrow\mathbb{R}$ is submodular if for all $j\in V$ and $A\subseteq B\subseteq V\setminus\{j\}$, the following holds:

[TABLE]

The multilinear extension $F:[0,1]^{V}\rightarrow\mathbb{R}$ of $f$ is defined as:

[TABLE]

Multilinear extensions are extensively used for maximizing their corresponding submodular set function and are known to be a special case of non-concave DR-submodular functions. The Hessian matrix of this class of functions has non-positive off-diagonal entries with zeros on its diagonal. It has been shown that for a large class of submodular set functions, their multilinear extension could be efficiently computed. Weighted matroid rank function, set cover function, probabilistic coverage function, graph cut function and concave over modular function are all examples of such submodular functions (see [15, 16] for more examples and details).

Non-convex/non-concave quadratic functions. Consider the quadratic function $F(x)=\frac{1}{2}x^{T}Hx+h^{T}x+c$. If the matrix $H$ is element-wise non-positive, $F$ would be a DR-submodular function. We use this class of non-concave DR-submodular functions for the numerical experiments.

See [12, 17] for more examples of continuous DR-submodular objective functions.

3 Problem Statement

The offline constrained optimization problem is as follows:

[TABLE]

where $\hat{x}_{i}^{T}\in\mathbb{R}_{+}^{m}$ is the $i$-th row and $x_{t}\in\mathbb{R}_{+}^{n}$ is the $t$-th column of the variable matrix $X\in\mathbb{R}_{+}^{n\times m}$, $\hat{c}_{i}^{T}\in\mathbb{R}_{+}^{m}$ and $c_{t}\in\mathbb{R}_{+}^{n}$ are the $i$-th row and $t$-th column of the cost matrix $C\in\mathbb{R}_{+}^{n\times m}$ respectively. For all $i\in[n]$, $H_{i}:K\to\mathbb{R}$, $K\subset\mathbb{R}_{+}^{m}$, is a differentiable monotone non-decreasing DR-submodular function which is zero at the origin (i.e., $H_{i}(0)=0$). For all $t\in[m]$, $F_{t}$ is a compact convex constraint set that contains the origin and $\|x\|_{2}\leq\lambda$ for all $x\in F_{t}$.

In the online setting, at step $t\in[m]$, $c_{t}$ and $F_{t}$ arrive online and the algorithm should choose $x_{t}\in F_{t}$ to maximize the overall objective function. Note that at each step $t\in[m]$, the function $H_{i}~{}\forall i\in[n]$ is only known over subsets of variables that have already arrived. Thus, we don’t have access to the objective function in advance.

The penalized formulation of problem 1 is the following:

[TABLE]

As an example, if for all $i\in[n]$,

[TABLE]

i.e., the concave indicator function of the interval $[0,1]$, the above two optimization problems are equivalent.

We aim to design differentiable, concave and monotone non-increasing penalty functions $G_{i}:\mathbb{R}_{+}\rightarrow\mathbb{R}~{}\forall i\in[n]$ and use them in our online algorithm such that the output doesn’t violate any of the linear packing constraints.

Online linear programming [5], the Adwords problem [2], single-unit combinatorial auction problem [18] and continuous generalization of online knapsack problem [19] are all special cases of this framework for linear objective functions.

Multiple applications of this framework are provided in Appendix A.

3.1 Dual Problem

The dual problem of the constrained problem 1 is as follows: (See Appendix B for the derivation)

[TABLE]

where for all $i\in[n]$, $z_{i}\in\mathbb{R}_{+}$, $\hat{y}_{i}^{T}\in\mathbb{R}^{m}$ is the $i$-th row of the dual matrix variable $Y$ and $y_{i,t}$ is the $(i,t)$-th entry of this matrix.

Karush–Kuhn–Tucker (KKT) conditions can be written as:

[TABLE]

where $G^{\prime}_{i}$ is the derivative of the scalar penalty function $G_{i}$. We remind the reader that we aim to design differentiable penalty functions $G_{i}~{}\forall i\in[n]$ and therefore, we have used $G^{\prime}_{i}$ in the KKT conditions.

We will use these KKT conditions to design the Generalized Sequential Algorithm.

4 Generalized Sequential algorithm and Competitive Ratio Analysis

4.1 Generalized Sequential Algorithm

Consider the Generalized Sequential algorithm below which outputs $\tilde{x}_{t}$ at each online step $t\in[m]$.

where for all $i\in[n]$, $t\in[m]$ and $k\in\{0,\dots,K\}$:

[TABLE]

In the above definitions, we have used the notation $[u]_{i}$ to denote the $i$-th entry of the vector $u$ and $\nabla_{t}$ denotes the $t$-th entry of the gradient vector.

At each online step $t\in[m]$, the algorithm performs a total of $K$ Frank-Wolfe updates in its inner loop where in each of these updates, a linear maximization problem over the set $F_{t}$ is solved. Note that in our applications, $F_{t}$ is usually a box constraint or the simplex and therefore, the corresponding linear maximization problem could be solved efficiently. See [20] for more details about using Frank-Wolfe for non-convex objectives.

The Generalized Sequential algorithm reduces to the Sequential algorithm in [7] for $K=1$ and hence the name. Additionally, this algorithm could be interpreted as the online counterpart of the offline Frank-Wolfe variant proposed in [12] for solving offline constrained continuous DR-submodular optimization problems. Note that at step $t\in[m],k\in[K]$, if we set \hat{y}_{i}^{*}=\nabla H_{i}\big{(}\omega_{i,t}(k-1)\big{)} and $z_{i}^{*}=-G^{\prime}_{i}(\hat{c}_{i}^{T}\omega_{i,t}(k-1))$, the update rule for $v_{t}(k)$ is similar to the KKT condition for $x_{t}^{*}$. In other words, the Generalized Sequential algorithm uses $\nabla H_{i}(\omega_{i,t}(k-1))$ and $-G^{\prime}_{i}(\hat{c}_{i}^{T}\omega_{i,t}(k-1))$ as the current estimate of $\hat{y}_{i}^{*}$ and $z_{i}^{*}$ respectively and using them, the algorithm obtains $v_{t}(k)$ to improve the estimate of $x_{t}^{*}$.

We define:

[TABLE]

${\rm ALG}$ and ${\rm P_{gseq}}$ are the objective value of problems 1 and 2 at the end of the algorithm respectively. Note that since $0\in F_{t}~{}\forall t\in[m]$, whenever $d_{t}(k-1)\preceq 0$, the algorithm would assign zero to $v_{t}(k)$ and therefore, ${\rm P_{gseq}}\geq 0$.

4.2 Competitive Ratio Analysis

First, we remind the reader that $H_{i}~{}\forall i\in[n]$ are DR-submodular and not necessarily concave. On the other hand, $G_{i}~{}\forall i\in[n]$ are concave penalty functions. In order to derive the competitive ratio, we make the following smoothness assumption about the functions:

Assumption 1: For all $i\in[n]$, functions $H_{i}$ and $G_{i}$ have an $L$-Lipschitz gradient, i.e., for all $x\in K$ and $u\in\mathbb{R}_{+}^{m}$ where $u\succeq 0$ or $u\preceq 0$, the following holds:

[TABLE]

Also, for all $x\in\mathbb{R}$ and $v\in\mathbb{R}$, we have:

[TABLE]

We also define the parameter $\alpha$ as follows:

Definition 4.1

For all $i\in[n]$, $\alpha_{H_{i}}$ is defined as:

[TABLE]

*Since $H_{i}$ is monotone non-decreasing, $0\leq\langle\nabla H_{i}(u),u\rangle$ holds. Additionally, because $H_{i}$ satisfies the DR property and $H_{i}(0)=0$, we have $\langle\nabla H_{i}(u),u\rangle\leq H_{i}(u)$. Thus, $-1\leq\alpha_{H_{i}}\leq 0$ always holds.

The definition above is inspired by the definition of $\alpha$ in [7]. The parameter $\alpha$ characterizes the curvature of the function. In fact, $\alpha$ of the multilinear extension of a submodular function and the total curvature of the underlying submodular set function are related as follows:

Remark 4.1

**Connection between total curvature of a submodular function and $\alpha$:

Recall that for a non-negative normalized monotone non-decreasing submodular function $f:2^{V}\rightarrow\mathbb{R}~{}\forall i\in[n]$, total curvature is defined as [21]:**

[TABLE]

If we denote the multilinear extension of this function by $F$, the following holds:

[TABLE]

See Appendix C for the proof.

If $n=1$, we design the penalty function $G_{1}$ as follows:

[TABLE]

If $n>1$, for all $i\in[n]$, we define the penalty function $G_{i}$ in the following:

[TABLE]

where for all $i\in[n]$, $U_{i}$ and $L_{i}$ are defined as follows:

[TABLE]

Roughly speaking, $U_{i}$ and $L_{i}$ are upper and lower bounds for the value-to-weight ratio of the items arriving online respectively. We are assuming that these upper and lower bounds are available offline to design the penalty functions. Our design for the penalty function for $n=1$ is inspired by the threshold function proposed by [19]. In the $n>1$ case, our penalty functions are inspired by the allocation rule of the primal-dual algorithm for the Adwords problem [3]. In both cases, the penalty functions are designed such that for $u\geq 1$, the algorithm assigns zero and thus, $G^{\prime}_{i}(1)=-U_{i}$ holds. Thus, the algorithm’s assignments would not violate the budget constraints.

If we denote the optimal values of the original constrained problem 1 and its dual problem 3 by ${\rm OPT}$ and ${\rm D}^{*}$ respectively, ${\rm OPT}\leq{\rm D}^{*}$ holds due to weak duality.

Now, we have all the required tools to obtain the competitive ratio bounds.

Theorem 4.1

For $n>1$, if Assumption 1 holds and $K\rightarrow\infty$, then for the Generalized Sequential algorithm, we have:

[TABLE]

This bound is tight in several known special cases. For the Adwords problem, since $U_{i}=L_{i}=1$ and $\alpha_{H_{i}}=0$ for all $i\in[n]$, competitive ratio of $1-\frac{1}{e}$ is obtained which is optimal [2]. Additionally, for online linear programming, considering that $\alpha_{H_{i}}=0~{}\forall i\in[n]$, we obtain \big{(}{\max_{i\in[n]}\ln(1+\frac{U_{i}(e-1)}{L_{i}})}\big{)}^{-1}\times(1-\frac{1}{e}) as the competitive ratio bound which is known to be optimal [7, 5].

**Proof ** See Appendix D for the proof.

Remark 4.2

For $n>1$, if we allow all the linear packing constraints to be violated by at most $\epsilon$, by modifying the penalty function for all $i\in[n]$ to

[TABLE]

competitive ratio improves to (1+\epsilon)\times\big{(}{\max_{i\in[n]}\big{\{}-(1+\epsilon)\alpha_{H_{i}}+\ln(1+\frac{U_{i}(e-1)}{L_{i}})\frac{e}{e-1}\big{\}}}\big{)}^{-1}

Theorem 4.2

For $n=1$, if Assumption 1 holds and $K\rightarrow\infty$, then for the Generalized Sequential algorithm, we have:

[TABLE]

For the online linear knapsack problem, since $\alpha_{H_{1}}=0$, competitive ratio of $\frac{1}{1+\ln(\frac{U_{1}}{L_{1}})}$ is obtained which is optimal [19] (note that because we allow fractional assignments, we do not need the small bids assumption to obtain the optimal competitive ratio).

**Proof ** See Appendix E for the proof.

Remark 4.3

For $n=1$, if we allow the linear packing constraint to be violated by at most $\epsilon$, modifying the penalty function to

[TABLE]

we obtain the improved competitive ratio of $\frac{1+\epsilon}{-(1+\epsilon)\alpha_{H_{1}}+\ln(\frac{U_{1}e}{L_{1}})}$

Theorems 4.1 and 4.2 provide the first competitive ratio bounds that generalize the results of [6, 7] for the concave case to general continuous DR-submodular objective functions which are not necessarily concave.

5 Experiments

We defined $F_{t}=\{x\in\mathbb{R}^{n}:0\preceq x\preceq\bf{1}\}$ for all $t\in[m]$ and we randomly generated monotone non-convex/non-concave quadratic functions of the form $F(x)=\frac{1}{2}x^{T}Hx+h^{T}x$ (see 2.1) where $H\in\mathbb{R}^{m\times m}$ is a random matrix with uniformly distributed non-positive entries in $[-100,0]$ and $h=-H^{T}\bf{1}$ to make the gradient non-negative. Therefore, the utility functions are of the form $F(x)=(\frac{1}{2}x-{\bf{1}})^{T}Hx$. We set the linear packing constraints to be of the form $Cx\preceq\bf{1}$ where $C\in\mathbb{R}^{n\times m}$ has uniformly distributed entries in $[0,1]$. We set $m=100$ and $K=50$. For all $i\in[n]$, the lower and upper bounds $L_{i}$ and $U_{i}$ were optimized by the input data. We ran the Generalized Sequential algorithm for both cases of $n=1$ and $n>1$ (note that the penalty function defined in these two cases were different) and in order to compute the competitive ratio, we divided the output of the algorithm to the offline optimal solution computed by the Frank-Wolfe variant algorithm of [12] with $K=50$. The average performance of the Generalized sequential algorithm over $10$ repeated experiments is summarized in Table 1. All codes were implemented in Python $3.7$ and the program was executed on a standard laptop computer ($2.30$GHz CPU, $16.0$GB Memory).

Table 1 shows that the output of the Generalized Sequential algorithm is not using all of the available budget which is natural in the adversarial input model. In other words, considering that no information about the online input is available, in order to attain a guaranteed competitive ratio, the algorithm needs to be overly cautious so that it does not miss valuable items that are arriving in the later steps due to exhausting all of the budget in the earlier stages.

6 Related Work

Offline submodular maximization. Consider the problem $\max_{x\in\mathcal{P}}F(x)$ where $F:\mathbb{R}^{m}\rightarrow\mathbb{R}$ is a non-negative monotone DR-submodular function and $\mathcal{P}$ is a down-closed convex set in the positive orthant. In [22], the special case of continuous relaxation of the discrete submodular function maximization problem subject to a matroid constraint is considered, the following variant of the Frank-Wolfe algorithm called the Continuous Greedy is used and a $(1-\frac{1}{e})$ approximation ratio is obtained. As it was mentioned in section 2.1, the multilinear extension of a submodular function satisfies the DR property.

[TABLE]

In this algorithm, $y(1)=\int_{0}^{1}v_{\rm max}(y(\tau))d\tau$ is the output. Our Generalized Sequential algorithm is in fact the online counterpart of the discretized Continuous Greedy algorithm (i.e., K is the number of discrete steps in our algorithm)

[12] obtained a similar approximation ratio for general continuous DR-submodular functions. [23] also exploited the same algorithm to obtain a $(1-\frac{1}{e})$ approximation ratio for submodular maximization subject to multiple linear constraints. Later on, the Continuous Greedy algorithm has been generalized to obtain approximation ratios for both monotone and non-monotone continuous submodular functions [12, 24, 25, 26, 27, 28]. See [29, 30] for a thorough overview of offline submodular maximization problems and algorithms.

Online knapsack problem. Consider the problem $\max_{x:c^{T}x\leq b}f(x)$ where $x=[x_{1},\dots,x_{m}]^{T}\in\{0,1\}^{m}$. In the online setting, at step $t\in[m]$, $t$-th item arrives and $c_{t}$ along with the value of the function $f$ over subsets of $\{1,\dots,t\}$ is revealed. The algorithm should decide whether to choose this item. [31] showed that in the adversarial setting, there exists no online algorithm achieving any non-trivial competitive ratio for this problem. [19] considered the case where $f(x)=d^{T}x;~{}d\in\mathbb{R}^{m}$ and proved that under the additional assumptions that for all $t\in[m]$, $c_{t}\ll b$ and $L\leq\frac{d_{t}}{c_{t}}\leq U$, there exists an algorithm that achieves the competitive ratio of $\frac{1}{1+ln(\frac{U}{L})}$ and is provably optimal. [10] generalized this algorithm for the case that the function $f$ is submodular and obtained a $\frac{1}{(1+\kappa_{f}+O(\epsilon))(1+ln(\frac{U}{L}))}$ competitive ratio where $L\leq\frac{f(t|S)}{c_{t}}\leq U~{}\forall t\in[m],S\subset\{1,\dots,m\}\setminus\{t\}$ and $\kappa_{f}$ is the total curvature of $f$ [21]. Note that if we apply our Generalized Sequential algorithm for $n=1$ to the multilinear extension of the function $f$ (which we denote by $F$) and allow fractional assignments of items, we obtain the competitive ratio $\frac{1}{1-\alpha_{F}+\ln(\frac{U}{L})}$ and because $\alpha_{F}\geq-\kappa_{f}$ holds by Remark 4.1, our bound improves upon the result of [10].

Submodular secretary problems. In this class of problems introduced by [32, 33], $m$ items are presented to the algorithm in random order. Upon arrival of an item, the algorithm should irrevocably decide whether to accept the current item. The goal is to maximize a monotone submodular function $f:\{0,1\}^{m}\rightarrow\mathbb{R}$ subject to cardinality, matching or linear packing constraints. See [34, 29] for a comprehensive overview of submodular secretary problems. Note that in the submodular secretary problem, the input is assumed to be stochastic while in our framework, the adversarial input model has been considered.

7 Conclusion

In this paper, we considered a class of online optimization problems, where the objective function is monotone DR-submodular under linear packing constraints. We specified various online discrete submodular problems whose continuous generalization could be cast in our framework (see Appendix A). We proposed the Generalized Sequential algorithm for solving such problems and we obtained competitive ratio bounds for this algorithm. Finally, we demonstrated the effectiveness of our algorithm through numerical experiments on a certain class of continuous DR-submodular functions.

Appendices

Appendix A Motivating Applications

There are a number of online budgeted discrete submodular problems whose continuous generalization could be cast in our framework. We have listed a number of these applications below:

Online Knapsack Constrained Continuous DR-submodular Maximization. In the discrete problem considered in [10], there is a ground set of elements $V$ and a budget constraint $b\in\mathbb{R}_{+}$. At step $t\in[m]$, an element $v\in V$ with the corresponding cost $c(v)\in\mathbb{R}_{+}$ arrives online and we should decide whether to choose $v$. The overall objective is as follows:

[TABLE]

where $f:2^{V}\rightarrow\mathbb{R}_{+}$ is a monotone non-decreasing submodular function and $V^{\prime}$ is the set of chosen elements. Note that at each step, value of the function is only known over subsets of items which have already arrived.

Consider the continuous relaxation of this problem where at each step, we are allowed to take a fraction of the arriving element. This problem could be formulated as:

[TABLE]

where $x=[x_{1},\dots,x_{m}]^{T}$, $c_{t}\in\mathbb{R}_{+}$ is the cost corresponding to the $t$-th arriving element and $F:[0,1]^{m}\rightarrow\mathbb{R}_{+}$ is the multilinear extension of the function $f$.

Online Generalized Maximum Coverage Problem. In this problem, there are $m$ subsets $C_{1},\dots,C_{m}$ of the ground set $V$ with corresponding costs $c_{1},\dots,c_{m}$ that are arriving one by one. At step $t\in[m]$, subset $C_{t}$ could be chosen with confidence level $x_{t}\in[0,1]$ and the set of covered elements when choosing $C_{t}$ with confidence $x_{t}$ is modeled with a monotone normalized covering function $p_{t}:[0,1]\to 2^{C_{t}}$ which is not known in advance and is revealed online. The goal is to choose subsets from $C_{1},\dots,C_{m}$ with confidence level to maximize the overall number of covered elements $|\bigcup_{t=1}^{m}p_{t}(x_{t})|$ while satisfying the budget constraint $\sum_{t=1}^{m}c_{t}x_{t}\leq b$. The problem could be formulated as follows:

[TABLE]

Online Continuous DR-submodular Welfare Maximization. In the submodular welfare problem, there is a set $M=\{1,\dots,m\}$ of $m$ items and a set $N=\{1,\dots,n\}$ of $n$ agents. Each agent $i\in N$ has a valuation function $f_{i}:2^{M}\rightarrow\mathbb{R}_{+}$ over subsets of items. Valuation functions are assumed to be submodular and monotone non-decreasing. In this problem, the goal is to partition the items among the agents as $S=(S_{1},\dots,S_{n})$, where $S_{s}\cap S_{t}=\emptyset~{}\forall s,t\in N$ and $\cup_{s=1}^{n}S_{s}=M$, in a way that the value of the partition $f(S)=\sum_{i=1}^{n}f_{i}(S_{i})$ is maximized [11]. Now, consider the continuous relaxation of this problem in the online setting: Each agent has a valuation function $F_{i}:[0,1]^{M}\rightarrow\mathbb{R}_{+}$ which is the multilinear extension of the submodular function $f_{i}$. At step $t\in M$, item $t$ arrives and the valuations of agents over subsets of items $\{1,\dots,t\}$ are accessible. The algorithm should assign item $t$ fractionally among the agents to maximize the aggregate valuation. The problem could be written as:

[TABLE]

where $\hat{x}_{i}=[x_{i1},\dots,x_{im}]^{T}$. Note that in this problem, there are no budget constraints.

Online DR-Submodular Generalized Assignment Problem. In this problem, there are $n$ bins and $m$ items. Each bin $i\in[n]$ has an associated collection of feasible sets given by the knapsack constraint $\mathcal{F}_{i}=\{S\subset[m]~{}:~{}\sum_{j\in S}c_{ij}\leq b_{i}\}$ and a monotone submodular valuation function $f_{i}:\{0,1\}^{m}\rightarrow\mathbb{R}_{+}$ which captures the diversity of the items in each bin. In the online setting, the set of items $t\in[m]$ arrive one by one and upon arrival of each item $t$, $c_{it}$ and values of the functions $f_{i}$ over subsets of $\{1,\dots,t\}$ for all $i\in[n]$ are revealed. The goal is to partition the items among the bins so as the aggregate valuation of the partition is maximized. If the valuation function $f_{i}$ is modular for all $i\in[n]$, this problem reduces to the Generalized Assignment Problem (GAP) [11]. Now, consider the continuous relaxation of this online problem where fractional assignments of items to bins are possible. The problem could be formulated as:

[TABLE]

where $\hat{x}_{i}=[x_{i1},\dots,x_{im}]^{T}$ and $F_{i}:[0,1]^{m}\rightarrow\mathbb{R}_{+}$ is the multilinear extension of the submodular valuation function $f_{i}$ of the $i$-th bin.

Appendix B Derivation of the Dual Problem

Let

[TABLE]

i.e., the convex indicator function of the set $F_{t}$.

Remember the offline constrained optimization problem:

[TABLE]

We derive the dual of problem 7 as follows:

[TABLE]

where $\hat{y}_{i}=[y_{i,1},\dots,y_{i,m}]^{T}$, $\sigma_{F_{t}}(u)=\sup_{w\in F_{t}}w^{T}u$ is the support function of the set $F_{t}$ and H_{i}^{*}(u)=\inf_{w}\big{(}w^{T}u-H_{i}(w)\big{)} is the concave conjugate function of $H_{i}$. Therefore, the dual problem is:

[TABLE]

Appendix C Connection between total curvature of a submodular function and $\alpha$

First, note that $F$, i.e., the multilinear extension of the discrete submodular function $f$, satisfies the DR property and $F(0)=0$. Since $F$ is linear in each of its arguments, we can write:

[TABLE]

Depending on whether $t\in S$ or not, one of the terms \big{(}f(S\cup\{t\})-f(S)\big{)} or \big{(}f(S)-f(S\setminus\{t\})\big{)} would be zero. So, by definition of total curvature of $f$, i.e., $\kappa_{f}$, we have:

[TABLE]

Combining 8 and 9, we have:

[TABLE]

Defining $\hat{x}_{t}=[x_{1},\dots,x_{t},0,\dots,0]^{T}$, we can write:

[TABLE]

Since $f(\{t\})=F(1_{t})=F(1_{t})-F(0)=\nabla_{t}F(0)$, using the DR property of the function $F$, $\nabla_{t}F(0)\geq\nabla_{t}F(\hat{x}_{t-1})$ and therefore, we have:

[TABLE]

Combining 10 and 11, we conclude:

[TABLE]

As a corollary, since $\alpha_{F}\in[-1,0]$ and $\kappa_{f}\in[0,1]$, if $\kappa_{f}=0$ (i.e., $f$ is modular), we can conclude that $\alpha_{f}=0$ as well.

Appendix D Proof of Theorem 4.1

For all $i\in[n]$, using the mean-value theorem, we have:

[TABLE]

where $u_{t}\in\mathbb{R}^{m}$ and $\omega_{i,t}(0)\preceq u_{t}\preceq\omega_{i,t}(K)$. Thus, we can write:

[TABLE]

Considering that $\|x\|_{2}\leq\lambda$ holds for all $x\in F_{t}$ and $t\in[m]$, we can write:

[TABLE]

where (a) is due to Assumption 1, (b) follows from $\|x\|_{2}\leq\lambda~{}\forall x\in F_{t},t\in[m]$, (c) uses the update rule of the Generalized Sequential algorithm and (d) is a result of subadditivity of the support function $\sigma_{F_{t}}$.

Using the DR assumption for $H_{i}$ and $G_{i}$, for all $i\in[n]$, $t\in[m]$ and $x\in F_{t}$, we can write: (we remind the reader that for scalar functions such as our concave penalty functions $G_{i}$, the DR property is equivalent to concavity)

[TABLE]

Taking supremum of 13 over all $x\in F_{t}$, we obtain:

[TABLE]

Therefore, we have:

[TABLE]

Now, we can use the definition of $\alpha$ to obtain:

[TABLE]

For all $i\in[n]$, using the definition of $G_{i}$ and defining $\gamma_{i}:=\ln(1+\frac{U_{i}(e-1)}{L_{i}})$, we have:

[TABLE]

Combining 12, 14, 15 and 16 along with $P_{\rm gseq}\geq 0$, we obtain:

[TABLE]

Therefore, if $K\to\infty$, the competitive ratio would be derived as \frac{{\rm ALG}}{{\rm D}^{*}}\geq\frac{1}{\max_{i\in[n]}\big{\{}-\alpha_{H_{i}}+\gamma_{i}\frac{e}{e-1}\big{\}}}.

Appendix E Proof of Theorem 4.2

Considering that G^{\prime}_{1}(u)=\ln(\frac{U_{1}e}{L_{1}})G_{1}(u)~{};u\geq\frac{1}{\ln\big{(}\frac{U_{1}e}{L_{1}}\big{)}}, combining 14 and 15 for $n=1$ along with $P_{\rm gseq}\geq 0$, we obtain:

[TABLE]

Therefore, if $K\to\infty$, the competitive ratio would be derived as $\frac{1}{1-\alpha_{H_{1}}+\ln(\frac{U_{1}}{L_{1}})}$.