Constrained Submodular Maximization via Greedy Local Search

TL;DR

This paper introduces a simple greedy local search algorithm that achieves near-optimal approximation ratios for maximizing monotone submodular functions under complex constraints like knapsacks and multiple matroids.

Contribution

The paper presents a new combinatorial algorithm combining greedy and local search for submodular maximization with knapsack and matroid constraints, extending known approximation guarantees.

Findings

Achieves a $rac{1 - e^{-2}}{2}$-approximation for knapsack and matroid constraints.

Extends to a $rac{1 - e^{-(k+1)}}{k+1}$ ratio for multiple matroid intersections.

Demonstrates the effectiveness of combining greedy algorithms with local search in constrained submodular maximization.

Abstract

We present a simple combinatorial $\frac{1 -e^{-2}}{2}$-approximation algorithm for maximizing a monotone submodular function subject to a knapsack and a matroid constraint. This classic problem is known to be hard to approximate within factor better than $1 - 1/e$. We show that the algorithm can be extended to yield a ratio of $\frac{1 - e^{-(k+1)}}{k+1}$ for the problem with a single knapsack and the intersection of $k$ matroid constraints, for any fixed $k > 1$. Our algorithms, which combine the greedy algorithm of [Khuller, Moss and Naor, 1999] and [Sviridenko, 2004] with local search, show the power of this natural framework in submodular maximization with combined constraints.

Figures

Tables

Table 1. Table 1 . Known results for maximizing a monotone submodular function subject to a knapsack and a matroid constraint. The ratio in  [ 5 ] becomes strictly better than the ratio in this paper by taking ε < 0.2 𝜀 0.2 \varepsilon<0.2 .

Approximation ratio	Running time
$1-1/e-\epsilon$	$\Omega \left(n^{\frac{1}{{\epsilon }^4}}\right)$ [5]
$\frac{1-e^{-2}}{2}\approx 0.432$	$\tilde{O}\left(n^6\right)$ This paper
$0.38$	$poly\left(n\right)$ [6]
$0.25$	$O\left(\frac{n}{{\epsilon }^2}{\log }^2\frac{n}{\epsilon }\right)$ [2]

Table 2. Table 2 . Comparison of “our work vs previous best” for monotone submodular maximization subject to a single knapsack and k > 1 𝑘 1 k>1 matroid constraints. The entries in the Previous Best column apply to the problem with O ​ ( 1 ) 𝑂 1 O(1) knapsack constraints.

$𝐤$	Our Work	Previous Best
2	$\frac{1-e^{-3}}{3}\approx$ 0.316	[14]
3	$\frac{1-e^{-4}}{4}\approx$0.245	[14]
5	$\frac{1-e^{-6}}{6}\approx$ 0.166	[14]
8	$\frac{1-e^{-9}}{9}\approx$ 0.111	[2]
9	$\frac{1-e^{-10}}{10}\approx$ 0.1	[2]

Equations

$$\underset{S\subseteq U}{\max }\{f(S):\text{leavevmode}S\in \mathcal{F}\text{mbox}and\sum _{u\in S}{c}_u\le B\}$$

$$\underset{S\subseteq U}{\max }\{f(S):\text{leavevmode}S\in \bigcap _{j=1}^k{\mathcal{F}}_j\text{mbox}and\sum _{u\in S}{c}_u\le B\}$$

$$\frac{{\sum }_{i=1}^P{\gamma }_i}{{\min }_{t\in \left[\mathrm{1..}P\right]}({\sum }_{i=1}^{t-1}{\gamma }_i+D{\gamma }_t)}\ge 1-{\left(1-\frac{1}{D}\right)}^P\ge 1-e^{-P/D}$$

$$f\left((Y\cup W\setminus \{w\})\cup \{u_{\ell }\}\right)-f(Y\cup W)$$

$$\le f(\{u_{\ell }\})-f(\varnothing )\le f(\{u_1\}),$$

$$f\left((Y\cup W\setminus \{w\})\cup \{u_{\ell }\}\right)-f(Y\cup W)$$

$$\le f(\{u_1\}\cup \{u_{\ell }\})-f(\{u_1\})$$

$$\le f(\{u_1,u_2\}=Y)-f(\{u_1\}),$$

$$2\left(f\left((Y\cup W\setminus \{w\})\cup \{u_{\ell }\}\right)-f(Y\cup W)\right)$$

$$f(S^{\ast })-f(S)$$

$$\le \sum _{x\in S^{\ast }\setminus S}\left(f(S\cup \{x\}\setminus \{b(x)\})-f(S\setminus \{b(x)\})\right)$$

$$\le \sum _{x\in S^{\ast }\setminus S}\left(f(S)-f(S\setminus \{b(x)\})\right).$$

$$f(S^{\ast })-f(S)$$

$$\le \sum _{i=1}^K\left(f(\{u_1,\dots ,u_i\})-f(\{u_1,\dots ,u_{i-1}\})\right)$$

$$\le f(\{u_1,\dots ,u_K\})\le f(S).$$

$$g(S^{\ast })\le g(S^t)+\sum _{u\in S^{\ast }\setminus S^t}(g(S^t\cup \{u\})-g(S^t)).$$

$$g(S^{\ast })-g(S^t)$$

$$\le \sum _{x\in S^{\ast }\setminus S^t}\left(g((S^t\setminus \{b_t(x)\})\cup \{x\})-g(S^t\setminus \{b_t(x)\})\right)$$

$$\le \sum _{x\in S^{\ast }\setminus S^t}\left(g(S^t)-g(S^t\setminus \{b_t(x)\})\right)+\left(g((S^t\setminus \{b_t(x)\})\cup \{x\})-g(S^t)\right)$$

$$\sum _{x\in C}(g(S^t)-g(S^t\setminus \{b_t(x)\}))$$

$$g(S^{\ast })$$

$$=2g(S^t)+\sum _{x\in S^{\ast }\setminus S^t}\left(f(S^t\setminus \{b_t(x)\}\cup \{x\})-f(S^t)\right)$$

$$\le 2g(S^t)+(B-c(Y)){\rho }_{t+1}.$$

$$g(S^{\ast })\le 2\left(g(S^t)+\frac{(B-c(Y))}{2}{\rho }_{t+1}\right).$$

$$g((S^{t^{\ast }}\setminus \{b_{t^{\ast }}(u_{t^{\ast }+1})\})\cup \{u_{t^{\ast }+1}\})=\sum _{\tau =1}^{t^{\ast }+1}(B_{\tau }-B_{\tau -1}){\rho }_{\tau }=\sum _{j=1}^{B^{\prime }}{\gamma }_j,$$

$$g(S^t)=\sum _{\tau =1}^t(B_{\tau }-B_{\tau -1}){\rho }_{\tau }=\sum _{j=1}^{B_t}{\gamma }_j.$$

$$\underset{s\in \left[\mathrm{1..}{B}^{\prime }\right]}{\min }\left\{\sum _{j=1}^{s-1}{\gamma }_j+\frac{B^{\prime \prime }}{2}{\gamma }_s\right\}$$

$$=\underset{t\in \left[\mathrm{1..}{t}^{\ast }\right]}{\min }\left\{g(S^t)+\frac{B^{\prime \prime }}{2}{\rho }_{t+1}\right\}.$$

$$\frac{g(S^{t^{\ast }}\setminus \{b_{t^{\ast }}(u_{t^{\ast }+1})\}\cup \{u_{t^{\ast }+1}\})}{g(S^{\ast })}$$

$$\ge \frac{1}{2}\left(1-(1-\frac{2}{B^{\prime \prime }}{)}^{B^{\prime }}\right)$$

$$\ge \frac{1}{2}\left(1-e^{-\frac{2B^{\prime }}{B^{\prime \prime }}}\right)\ge \frac{1}{2}(1-e^{-2})$$

$$f(S^{t^{\ast }})$$

$$=f(Y)+g((S^{t^{\ast }}\setminus \{b_{t^{\ast }}(u_{t^{\ast }+1})\})\cup \{u_{t^{\ast }+1}\})$$

$$-\left(g((S^{t^{\ast }}\setminus \{b_{t^{\ast }}(u_{t^{\ast }+1})\})\cup \{u_{t^{\ast }+1}\})-g(S^{t^{\ast }})\right)$$

$$=f(Y)+g((S^{t^{\ast }}\setminus \{b_{t^{\ast }}(u_{t^{\ast }+1})\})\cup \{u_{t^{\ast }+1}\})$$

$$-\left(f((S^{t^{\ast }}\setminus \{b_{t^{\ast }}(u_{t^{\ast }+1})\})\cup \{u_{t^{\ast }+1}\})-f(S^{t^{\ast }})\right)$$

$$\ge f(Y)+\frac{1}{2}(1-e^{-2})g(S^{\ast })-\frac{f(Y)}{2}$$

$$=\frac{f(Y)}{2}-\frac{f(Y)}{2}(1-e^{-2})+\frac{1}{2}(1-e^{-2})f(S^{\ast })$$

$$\ge \frac{1}{2}(1-e^{-2})f(S^{\ast })$$

$$f(S^{\ast })$$

$$f(S^{\ast })-f(S)$$

Full text

Our algorithms, which combine the greedy algorithm of [Khuller, Moss and Naor, 1999] and [Sviridenko, 2004] with local search, show the power of this natural framework in submodular maximization with combined constraints.

1. Introduction

A set function $f\!\!:\!2^{U}\rightarrow I\!\!R$ is submodular if for every $R,T\subseteq U$, $f(R)+f(T)\geq f(R\cup T)+f(R\cap T)$. Such functions are ubiquitous in diverse fields, most notably in combinatorial optimization, operations research and economics. An equivalent definition of submodularity, which is perhaps more intuitive, refers to its diminishing returns: $f(R\cup\{u\})-f(R)\leq f(T\cup\{u\})-f(T)$, for any $T\subseteq R\subseteq U$, and $u\in U\setminus R$. The concept of diminishing returns is widely used in economics, often leading to submodular utility functions. Submodular functions also provide a unifying framework which captures optimization problems that have been studied earlier, such as Min Cut, Max Cut, Multiway Cut, Maximum Coverage, the Generalized Assignment Problem, or the Separable Assignment Problem.

In these settings, the goal is to optimize a submodular function subject to certain constraints. In fact, in many cases the constraints at hand are quite simple, such as knapsack constraints, matroid constraints, or a combination of the two. Such submodular maximization problems naturally arise in advertising campaigns [1], combinatorial auctions [8, 33], social networks [22, 23], and document/corpus summarization [31, 25].

We consider first the classic problem of maximizing a monotone submodular function subject to a knapsack and a matroid constraint. Formally, let $f$ be a non-negative monotone submodular function of a ground set $U$. Throughout the paper, we assume that $f$ is given via a value oracle; that is, given a set $S\subseteq U$, the oracle returns $f(S)$. Let ${{\cal F}}\subseteq 2^{U}$ be a family of subsets of $U$, and ${{\cal M}}=(U,{{\cal F}})$ a matroid defined over $U$ and ${{\cal F}}$ (see Section 2 for the formal definition).

The subsets in the collection ${{\cal F}}$ are called independent sets. Suppose that each element $u\in U$ has a nonnegative size $c_{u}$, and let $B$ be a given size bound. Our goal is to maximize $f(S)$ over all subsets $S\subseteq U$, such that $S$ is an independent set of ${{\cal M}}$, and the total size of the elements in $S$ is bounded by $B$, i.e.,

[TABLE]

We further consider a generalization of (1) in which we are given $k$ collections of independent sets, ${{\cal F}}_{1},\ldots,{{\cal F}}_{k}$, each containing subsets of $U$, and the corresponding matroids ${{\cal M}}_{j}=(U,{{\cal F}}_{j})$, $j=1,\ldots,k$. The goal is to maximize $f(S)$ over all subsets $S$ of $U$ which are independent sets in $\bigcap_{j=1}^{k}{{\cal F}}_{j}$. Formally, the optimization problem is

[TABLE]

There is a beautiful line of research in this area. For a long time, the only known work had been a sequence of papers by Fisher, Nemhauser and Wolsey [30, 16, 29], showing that a greedy algorithm achieves a ratio of $1-1/e$ to the optimum for maximizing a monotone submodular function under a cardinality constraint,111The special case of a knapsack constraint where $c_{u}=1$ for all $u\in U$. with a matching hardness of approximation result in the oracle model. The paper [16] shows that simple local search yields a ratio of $1/2$ when the function is maximized under a matroid constraint. A greedy algorithm was shown to achieve a ratio of $\frac{1}{k+1}$ for this problem with $k$ matroid constraints.

The hardness of approximation within ratio better than $1-1/e$, already for Maximum Coverage (i.e., maximizing a linear function under cardinality constraint), follows from a result of Feige [9]. This has led to an ongoing research aiming to develop approximation algorithms which come close to this bound. The seminal paper of Khuller, Moss and Naor [24] achieved the ratio of $1-1/e$ for Budgeted Maximum Coverage, using a greedy algorithm. Their result inspired the later work of Sviridenko [32], presenting a simple greedy algorithm for maximizing a monotone submodular function subject to a knapsack constraint. For a single matroid constraint, there are several algorithms achieving the ratio $1-1/e$ via continuous greedy and multilinear relaxations [33, 4] as well as other advanced techniques [15, 2].

For all of these cases the best known approximation ratios are essentially the best possible, with many of the algorithms easy to employ in practical scenarios; yet, already for maximizing a monotone submodular function subject to a knapsack and a matroid constraint, attempting to approach the ratio $1-1/e$ within additive of $\varepsilon$, for any fixed $\varepsilon>0$, requires existing algorithms to perform $\Omega(n^{\mathrm{poly}(1/\varepsilon)})$ steps [5] (see also [18]). The dependence on $1/\varepsilon$ in the exponent renders these algorithms impractical even for $\varepsilon=1/2$. The next best ratio of $0.38$, obtained via multilinear relaxations and contention resolution schemes [6], deviates from the known hardness of approximation bound of $1-1/e$ almost by factor of $2$.222The bound of $1-1/e$ follows from the known inapproximability results for each constraint alone [9, 12]. A fast algorithm of [2] achieves the ratio $0.25$. Similarly, for a single knapsack constraint and $k$ matroid constraints, where $2\leq k\leq 5$, the best known ratio is $h(k,\varepsilon)=\max_{0\leq b\leq 1}(1-e^{-b}-\epsilon)[(1-e^{-b})/b]^{k}$, due to  [14] (see also [11] and Table 2). For fixed $k>5$, the best ratio is $\frac{1}{k+3+\varepsilon}$ derived in [2].333The algorithm of [2] achieves these bounds in almost linear time, for a class of more general instances. The best known hardness of approximation result is $O(\frac{\log k}{k})$, due to [21], for the intersection of $k>1$ matroid constraints.

In this work, we amend the above state of affairs by obtaining practical algorithms whose approximation ratios are closer to the known hardness of approximation results. We note that some of the previous algorithms have better running times, while the results of [5, 18] lead to better bounds (albeit at high computational cost). Yet, as detailed below, a notable advantage of our algorithms is in being extremely simple.

While we focus in this paper on maximizing a monotone submodular function under a single knapsack and (single or multiple) matroid constraints, there are known results for extension to intersection of $k$-system and $\ell$ knapsack constraints (e.g., [17, 19, 20, 2, 6]), and for maximizing a non-monotone submodular function [10, 28, 27, 20, 13, 14, 26, 6, 12]. For unconstrained submodular maximization, the best known result is a randomized linear time $1/2$-approximation algorithm by Buchbinder et al. [3].

**Our Contribution: ** We present combinatorial algorithms for maximizing a monotone submodular function subject to a knapsack and $k$ matroid constraints, yielding approximation ratio of $\frac{1-e^{-(k+1)}}{k+1}$, for any fixed $k\geq 1$. The algorithms are based on the greedy approach of [24, 32] combined with local improvement steps which allow $k$-swaps (see, e.g., [27]). These easy features make the algorithms highly intuitive in tackling our problems. For the single matroid case, we show that our algorithm can be implemented in ${\tilde{O}}(n^{6})$ time, where $n=|U|$ is the cardinality of the ground set.444The notation $\tilde{O}$ “hides” poly-logarithmic factors. We note that the greedy algorithm of Sviridenko [32], that yields the best possible ratio of $1-1/e$ for maximizing a submodular function subject to a knapsack constraint, has running time $O(n^{5})$. For a single knapsack and $k$ matroid constraints, where $k>1$ is fixed, the running time of our algorithm is $n^{O(k)}$. Table 1 gives the known results for the special case where $k=1$, along with the running times of the algorithms. Table 2 compares our approximation ratios with best previous results for several values of $k>1$.

Our algorithms handle the combined knapsack and matroid constraints by applying greedy swaps. While the greedy property guarantees that we do not exceed the knapsack constraint without collecting enough value, the swaps maintain the independence of the solution set given the matroid constraint. Our algorithms for single and multiple matroid constraints (outlined in Alg. 1 and Alg. 2, respectively) proceed in the same fashion. The only difference is that in the multiple matroid case, we replace the ‘swaps’ with ‘$k$-swaps’; a $k$-swap may involve the deletion of up to $k$ elements from the solution set, while adding to the set a single element. To the best of our knowledge, this notion of greedy swaps is used here for the first time.

2. A Single Matroid Constraint

We first study the problem where we have a knapsack constraint and a single matroid constraint (as defined in Eq. (1)).

We start with some definitions and notation. For a subset of elements $U^{\prime}\subseteq U$, let $c(U^{\prime})$ denote the total size of the elements in $U^{\prime}$, i.e., $c(U^{\prime})=\sum_{u\in U^{\prime}}c_{u}$.

For notational convenience, we consider $\phi$ as a dummy element not in the ground set $U$. We assume that $c_{\phi}=0$, and for any set $U^{\prime}\subseteq U$, $f(U^{\prime}\cup\{\phi\})=f(U^{\prime})$. Given a subset $U^{\prime}\subseteq U$, where $U^{\prime}$ is independent, i.e., $U^{\prime}\in{{\cal F}}$, a pair of elements $(x,y)$, where $x\in U\setminus U^{\prime}$ and $y\in U^{\prime}\cup\{\phi\}$, is a swap if $(U^{\prime}\setminus\{y\})\cup\{x\}$ is independent, i.e., $(U^{\prime}\setminus\{y\})\cup\{x\}\in{{\cal F}}$. Let $L(U^{\prime})$ denote the collection of swaps that involve elements from $U^{\prime}$. The marginal profit density of a swap $(x,y)$ is given by $\rho_{(x,y)}=\frac{f((U^{\prime}\setminus\{y\})\cup\{x\})-f(U^{\prime})}{c_{x}}$.

2.1. The Algorithm

Let ${S^{*}}=\{u_{1},u_{2},\ldots,u_{p}\}$ be an optimal solution of the given instance, ordered such that $u_{i+1}$ is the element with maximum marginal profit with respect to the prefix $\{u_{1},u_{2},\ldots,u_{i}\}$. We assume that $|{S^{*}}|\geq 2$, since in case ${S^{*}}$ consists of a single element, this element can be guessed to yield an optimal solution. The algorithm starts by guessing $Y=\{u_{1},u_{2}\}$, the two elements with largest marginal profits in ${S^{*}}$.

We set the initial solution to consist of the set $Y$. The algorithm then applies iterations of ‘greedy swaps’ that expand the solution while maintaining the knapsack and the matroid constraint.

We note that for ease of understanding the pseudocode of 1-MatKnap (in Algorithm 1) does not guarantee that the algorithm terminates in polynomial time. However, in Section 2.3 we discuss a modified version of our algorithm that runs in polynomial time with no harm to the approximation ratio.

2.2. Analysis

We now show that Algorithm 1-MatKnap yields an approximation ratio that is close to the optimal of $1-e^{-1}$ known for submodular maximization with either a knapsack or a matroid constraint. Formally, our main result is the following.

Theorem 2.1**.**

1-MatKnap is a $\frac{1-e^{-2}}{2}$-approximation algorithm for submodular maximization subject to a knapsack and a matroid constraint.

The following simple lemmas will be useful in the proof of Theorem 2.1. Let ${{\cal F}}$ be a collection of subsets of $U$, and ${{\cal M}}=(U,{{\cal F}})$ a matroid defined on $U$. Recall that ${{\cal M}}$ satisfies the following (see, e.g., [7]).

1. $(i)$

Non-emptiness: The empty set is in ${{\cal F}}$ (thus, ${{\cal F}}$ is not itself empty). 2. $(ii)$

Hereditary property: If a set $R$ is in ${{\cal F}}$ then every subset of $R$ is also in ${{\cal F}}$. 3. $(iii)$

Exchange property: If $R$ and $T$ are two sets in ${{\cal F}}$, where $|R|>|T|$, then there is an element $r\in R\setminus T$ such that $T\cup\{r\}$ is in ${{\cal F}}$.

We note that all maximal independent sets (or bases) of a matroid have the same cardinality. The first lemma follows directly from the above Exchange property.

Lemma 2.2**.**

Let $B_{1},B_{2}$ be two bases of a matroid ${{\cal M}}=(U,{{\cal F}})$, then for any $x\in B_{2}\setminus B_{1}$, there exists $y\in B_{1}\setminus B_{2}$ such that $(B_{1}\setminus\{y\})\cup\{x\}$ is a base of ${{\cal M}}$.

Lemma 2.3**.**

Given two independent sets $S,T\in{{\cal F}}$, there exists a mapping $b:T\setminus S\rightarrow(S\setminus T)\cup\{\phi\}$, such that $(S\setminus\{b(u)\})\cup\{u\}\in{{\cal F}}$, for all $u\in T\setminus S$, and $|b^{-1}(y)|\leq 1$, for all $y\in S\setminus T$.

Proof 2.4**.**

Let ${{{\tilde{B}}^{*}}}$ be a base of ${{\cal M}}$, such that $T\subseteq{{{\tilde{B}}^{*}}}$. By the Exchange property, $|{{{\tilde{B}}^{*}}}|-|S|$ elements from ${{{\tilde{B}}^{*}}}\setminus S$ can be added to the set $S$ to obtain a base of ${{\cal M}}$. Denote this base by ${\tilde{B}}$. For each element $x\in T\cap({\tilde{B}}\setminus S)$, define $b(x)=\phi$.

We now iterate over all elements in $T\setminus{\tilde{B}}\subseteq{{{\tilde{B}}^{*}}}\setminus{\tilde{B}}$. Let $x\in T\setminus{\tilde{B}}$ be the current element. By Lemma 2.2, there exists $y\in{\tilde{B}}\setminus{{{\tilde{B}}^{*}}}$, such that $({\tilde{B}}\setminus\{y\})\cup\{x\}$ is a base of ${{\cal M}}$. Note that $y$ must be in $S\setminus T$, since ${\tilde{B}}\setminus(S\setminus T)\subseteq{{{\tilde{B}}^{*}}}$. We set ${\tilde{B}}=({\tilde{B}}\setminus\{y\})\cup\{x\}$ and $b(x)=y$, and iterate on the next element in $T\setminus{\tilde{B}}$. By our construction, no two elements in $T\setminus{\tilde{B}}$ are mapped to the same element in $S\setminus T$.

The next lemma is due to Wolsey [34].

Lemma 2.5**.**

Given two positive integers, $P$ and $D$, and a set of real-valued non-negative numbers $\gamma_{i}$, $i=1,\ldots,P$,

[TABLE]

The following generalizes a result of Sviridenko [32]. We include the proof here for completeness.

Lemma 2.6**.**

Given an element $u_{\ell}\in{{S^{*}}}$, $\ell\geq 3$, and a subset $W\subseteq U\setminus\{u_{1},u_{2},u_{\ell}\}$, for any swap $(u_{\ell},w)$, such that $w\in W\cup\{\phi\}$, $f((Y\cup W\setminus\{w\})\cup\{u_{\ell}\})-f(Y\cup W)\leq f(Y)/2.$

Proof 2.7**.**

The ordering of the elements of ${{S^{*}}}$, and the fact that $f(\cdot)$ is submodular, monotone, and non-negative imply that for $u_{\ell}\in{{S^{*}}}$, where $\ell\geq 3$ and the subsets $Y$ and $W$, the following inequalities are satisfied:

[TABLE]

and

[TABLE]

Summing the two inequalities, we have that

[TABLE]

Proof 2.8** (Proof of Theorem 2.1).**

*Suppose we have guessed $u_{1}$ and $u_{2}$ correctly. Let $S\subseteq U$ denote the subset output by the algorithm.

By Lemma 2.3, there exists a mapping $b:{{S^{*}}}\setminus S\rightarrow(S\setminus{S^{*}})\cup\{\phi\}$, such that $(S\setminus\{b(u)\})\cup\{u\}\in{{\cal F}}$, for all $u\in{{S^{*}}}\setminus S$, and $|b^{-1}(y)|\leq 1$, for all $y\in S\setminus{{S^{*}}}$.*

Throughout the analysis, we refer to iterations of the outer loop (Line 4.) in Algorithm 1. Thus, each iteration, except maybe the last one, consists of a single greedy swap. We distinguish between two cases, based on the last iteration of the algorithm.

Case 1:* For all elements $x\in{S^{*}}\setminus S$, the swap $(x,b(x))$ was rejected in the last iteration because it satisfied $\rho_{(x,b(x))}\leq 0$, and not because $c_{x}-c_{b(x)}+c(S)>B$. That is, the swap $(x,b(x))$ did not violate the knapsack constraint.*

From Case 1 assumption, it follows that no swap $(x,b(x))$, for an element $x\in{S^{*}}\setminus S$, could increase the value of $f(\cdot)$. Hence, we have that

[TABLE]

The first and second inequalities follow from submodularity, and the third from our assumption about the swaps. Recall that for every $y\in S\setminus{S^{*}}$ we have $|b^{-1}(y)|\leq 1$, thus the (multi) set $\{b(x)\}_{x\in{S^{*}}\setminus S}$ does not contain any element from $S\setminus{S^{*}}$ with multiplicity higher than 1. Let $\{b(x)\}_{x\in{S^{*}}\setminus S}\setminus\phi=\{u_{1},\ldots,u_{K}\}$, where $K\leq|{S^{*}}\setminus S|$ and the indices are assigned arbitrarily. Then, by (2.8), and using submodularity and monotonicity, we have

[TABLE]

Hence, in this case, we have that 1-MatKnap yields a $\frac{1}{2}$-approximation to the optimum.

Case 2:* At least one swap $(x,b(x))$, for $x\in{S^{*}}\setminus S$, considered in the last iteration, violated the knapsack constraint; namely, $c_{x}-c_{b(x)}+c(S)>B$.*

Denote by $S^{\ell}$ the subset of elements in the solution after iteration $\ell$ of the algorithm, for $\ell\geq 0$, where $S^{0}=Y$. Let $b_{\ell}:{S^{*}}\setminus S^{\ell}\rightarrow(S^{\ell}\setminus{S^{*}})\cup\{\phi\}$ be the mapping guaranteed by Lemma 2.3. We note that in all iterations except the last the algorithm performs a swap $(x_{t},y_{t})$ (where $y_{t}$ may be $\phi$). Denote $\rho_{(x_{t},y_{t})}$ by $\rho_{t}$.

Let $t^{*}+1$ be the first iteration in which a swap $(u_{t^{*}+1},b_{t^{*}}(u_{t^{*}+1}))$, for $u_{t^{*}+1}\in{S^{*}}\setminus{S^{t}}$, was considered in iteration $t^{*}+1$ and was rejected since it violated the knapsack constraint. Clearly, such an iteration exists by the assumption of Case 2. By the choice of $t^{*}$, for all $t=0,\ldots,t^{*}-1$, for every $x\in{S^{*}}\setminus{S^{t}}$ such that $(x,b_{t}(x))$ is a swap, we must have $\rho_{(x,b_{t}(x))}\leq\rho_{t+1}$.

Define the function $g(S)=f(S)-f(Y)$. Since $f(\cdot)$ is submodular and monotone, so is $g(\cdot)$. Hence, for all $t=0,\ldots,t^{*}$,

[TABLE]

Using submodularity, we can now write:

[TABLE]

Claim 1**.**

$\displaystyle{\sum_{x\in{{{S^{*}}}\setminus{{S^{t}}}}}}(g({{S^{t}}})-g({{S^{t}}}\setminus\{b_{t}(x)\}))\leq g({{S^{t}}})$.

Proof 2.9**.**

Clearly, for any $x\in{{{S^{*}}}\setminus{{S^{t}}}}$ such that $b_{t}(x)=\phi$, we have $g({{S^{t}}})-g({{S^{t}}}\setminus\{b_{t}(x)\})=0$. Thus, we need only to consider the sum over the subset $C\subseteq{{{S^{*}}}\setminus{{S^{t}}}}$, such that for any $x\in C$, $b_{t}(x)\neq\phi$. As noted above, the (multi) set $\{b_{t}(x)\}_{x\in C}$ does not contain any element from ${S^{t}}\setminus{S^{*}}$ with multiplicity higher than 1. Let $\{b_{t}(x)\}_{x\in C}=\{u_{1},\ldots,u_{K}\}$, where $K=|C|$, and the indices are assigned arbitrarily. Then, similar to Inequality (2.8),

[TABLE]

Using Claim 1, we have for any $t=0,\ldots,t^{*}-1$,

[TABLE]

The last inequality is due to the following:

1. (a)

By our choice of $t^{*}$, for any swap $(x,b_{t}(x))$, $f({{S^{t}}}\setminus\{b_{t}(x)\}\cup\{x\})-f({{S^{t}}})\leq c_{x}{{\rho_{t+1}}}$. 2. (b)

$\sum_{x\in{{S^{*}}}\setminus{{S^{t}}}}c_{x}\leq B-c(Y)$.

Now, from the above discussion, we have for any $t=0,\ldots,t^{*}-1$,

[TABLE]

For $t=1,\ldots,t^{*}$, let $B_{t}=\sum_{\tau=1}^{t}(g(S^{\tau})-g(S^{\tau-1}))/\rho_{\tau}$, and $B_{0}=0$. Note that $(g(S^{\tau})-g(S^{\tau-1}))/\rho_{\tau}=c_{x_{\tau}}$. Since the algorithm performs swaps, we have that $B_{t}\geq c(S^{t})$. Also, let $\rho_{t^{*}+1}=\rho_{(u_{t^{*}+1},b_{t^{*}}(u_{t^{*}+1}))}$, and $B_{t^{*}+1}=B_{t^{*}}+c_{u_{t^{*}+1}}$ We note that, by the definition of $u_{t^{*}+1}$, we have that $B^{\prime}=B_{t^{*}+1}>B-c(Y)=B^{\prime\prime}$. For $j=1,\ldots,B_{t^{*}+1}$, let $\gamma_{j}={\rho_{t}}$ when $j=B_{t-1}+1,\ldots,B_{t}$. Using the above notation, we have that

[TABLE]

and for $t=1,\ldots,t^{*}$,

[TABLE]

Now, we note that

[TABLE]

Using Lemma 2.5 and Inequality (7), we have

[TABLE]

Finally, using Lemma 2.6,

[TABLE]

2.3. Running Time Analysis

We note that the running time of Algorithm 1-MatKnap, as described above, may not be polynomial, since the number of greedy swaps cannot be bounded polynomially. Below, we show how the algorithm can be implemented in polynomial-time with no harm to the approximation ratio.

Theorem 2.10**.**

Algorithm 1-MatKnap can be modified to run in ${\tilde{O}}(n^{6})$ time.

Proof 2.11**.**

To guarantee polynomial running time, we modify the greedy swaps as follows. Fix some $\varepsilon>0$ (to be set later). Whenever an improvement is encountered, a swap is performed only if the value of $f(\cdot)$ increases by at least a factor of 1+$\frac{\varepsilon}{n^{2}}$ as a result of the swap. (We note that this modification applies only to “real” swaps, and not to swaps with the dummy element $\phi$.) Let ${S^{*}}=\{u_{1},\ldots,u_{p}\}$, then by Lemma 2.6,

[TABLE]

Hence, the overall number of swaps is bounded by $\frac{n^{2}}{\epsilon}\log n$. We show next that we can find a constant $\varepsilon>0$ that would not degrade the approximation ratio of the algorithm. Consider the two cases in the proof of Theorem 2.1. In Case 1, after the modification Inequality (2.8) becomes

[TABLE]

Thus, we get $\frac{1}{2+\epsilon n^{-1}}$-approximation. Since the approximation factor in Case 2 is $\frac{1}{2}(1-e^{-2})$, the overall approximation remains unchanged, as long as $\frac{1}{2+\epsilon n^{-1}}\geq\frac{1}{2}(1-e^{-2})$, which implies $\frac{\epsilon}{n}\leq\frac{2}{e^{2}-1}$.

Now, consider Case 2 of Theorem 2.1. After the modification, Inequality (3.5) becomes

[TABLE]

Note that for any $x\in{S^{*}}\setminus S$, if $b_{t}(x)=\phi$ then $\rho_{(x,b_{t}(x))}\leq\rho_{t+1}$, otherwise, i.e. $b_{t}(x)\neq\phi$, either $g({{S^{t}}}\setminus\{b_{t}(x)\}\cup\{x\})\leq\left(1+\frac{\epsilon}{n^{2}}\right)g({{S^{t}}})$ or $\rho_{(x,b_{t}(x))}\leq\rho_{t+1}$. The set $C$ contains all the elements for which the first inequality is satisfied. Inequality (2.11) implies modified Inequality (2.8):

[TABLE]

Proceeding to propagate this modification, we finally get

[TABLE]

It follows that the approximation ratio remains $\frac{1}{2}(1-e^{-2})$, as long as $(\epsilon n^{-1})(1-e^{-2})f({{S^{*}}})\leq 2f(Y)e^{-2}$. By (10), we have $nf(Y)\geq 2f({{S^{*}}})$; thus, this holds for $\epsilon\leq\frac{4}{e^{2}-1}$.

We now turn to analyze the running time. We have $O(n^{2})$ guesses of the set $Y$. For each such guess, we have overall $\tilde{O}(n^{2})$ successful swaps. (Note that there are $O(n)$ swaps in which $\phi$ is involved.) We have overall $\tilde{O}(n^{2})$ swaps, each requires $O(n^{2})$ time. We conclude that the overall running time is $\tilde{O}(n^{6})$.

We remark that a factor of $n$ can be saved in the running time at the expense of reducing the approximation ratio by $O(\varepsilon)$. Specifically, we can perform the swaps only if the multiplicative improvement is 1+$\frac{\varepsilon}{n}$.

3. $k$ Matroid Constraints

In this section we extend our algorithm for a single matroid constraint to handle a single knapsack constraint and $k$ matroid constraints, for any fixed $k>1$; namely, we give an approximation algorithm for problem (2).

3.1. Algorithm

A key operation in our algorithm for $k$ matroid constraints is $k$-swap, in which we add to the solution set a single element and eliminate up to $k$ elements. Denote the collection of subsets of $U$ of size at most $k$ by $[U]^{\leq k}$; note that $\emptyset\in[U]^{\leq k}$. Given a subset $U^{\prime}\subseteq U$, $x\in U\setminus U^{\prime}$ and ${\bar{y}}\in[U^{\prime}]^{\leq k}$, $(x,{{\bar{y}}})$ is a k-swap if $(U^{\prime}\setminus{\bar{y}})\cup\{x\}$ is independent, i.e., $(U^{\prime}\setminus{\bar{y}})\cup\{x\}\in\bigcap_{j=1}^{k}{{\cal F}}_{j}$.

Let $L_{k}(U^{\prime})$ denote the collection of $k$-swaps. The marginal profit density of a $k$-swap $(x,{{\bar{y}}})$ is given by $\rho_{(x,{{\bar{y}}})}=\frac{f((U^{\prime}\setminus{{\bar{y}}})\cup\{x\})-f(U^{\prime})}{c_{x}}$.

**Overview: ** Similar to 1-MatKnap, Algorithm $k$-MatKnap modifies the solution set iteratively, while increasing the objective function value and maintaining the knapsack and matroid constraints. The main change is in replacing swaps with $k$-swaps. A pseudocode of $k$-MatKnap is given in Algorithm 2.

3.2. Analysis

We first analyze the performance ratio of $k$-MatKnap.

Theorem 3.1**.**

For any fixed $k\geq 1$, $k$-MatKnap is a $\frac{1-e^{-(k+1)}}{k+1}$-approximation algorithm for problem (2).

We use in the proof the next lemmas.

Lemma 3.2**.**

Given two independent sets $S,T\in\bigcap_{j=1}^{k}{{\cal F}}_{j}$, there exists a mapping of the elements in $T\setminus S$ to $[S\setminus T]^{\leq k}$ (namely, subsets of $S\setminus T$ of size at most $k$, including the empty set), such that each element $u\in S\setminus T$ appears in at most $k$ subsets.

Proof 3.3**.**

For a matroid ${{\cal M}}_{j}$, $1\leq j\leq k$, since $S,T\in{{\cal F}}_{j}$ are independent sets, by Lemma 2.3 there exists a mapping $b_{j}:T\setminus S\rightarrow(S\setminus T)\cup\{\phi\}$, such that $(S\setminus b_{j}(x))\cup\{x\}\in{{\cal F}}_{j}$, for all $x\in T\setminus S$. We use the mappings $b_{j}$ to define the mapping $b:T\setminus S\rightarrow[S\setminus T]^{\leq k}$ as follows. For each element $x\in T\setminus S$, define $b(x)=\bigcup_{j=1}^{k}\{b_{j}(x)\}\cap(S\setminus T)$. The $k$-swaps are defined accordingly. By Lemma 2.3, for each element $u\in S\setminus T$, and for $1\leq j\leq k$, $|b_{j}^{-1}(u)|\leq 1$. It follows that each element $u\in S\setminus T$ appears in at most $k$ mapped subsets.

The proof of the next lemma is similar to the proof of Lemma 2.6 (details omitted).

Lemma 3.4**.**

Given an element $u_{\ell}\in{{S^{*}}}$, $\ell\geq 3$, and a subset $W\subseteq U\setminus\{u_{1},u_{2},u_{\ell}\}$, for any $k$-swap $(u_{\ell},{\bar{w}})$, where ${\bar{w}}\in[W]^{\leq k}$, $f((Y\cup W\setminus{\bar{w}})\cup\{u_{\ell}\})-f(Y\cup W)\leq f(Y)/2.$

Proof 3.5** (Proof of Theorem 3.1).**

*Suppose we have guessed $u_{1}=\mbox{argmax}_{u\in{S^{*}}}f(\{u\})$ and

$u_{2}=\mbox{argmax}_{u\in{S^{*}}\setminus{\{u_{1}\}}}f(\{u,u_{1}\})-f(\{u_{1}\})$ correctly. Let $S\subseteq U$ denote the subset output by the algorithm. By Lemma 3.2, there exists a mapping $b:{{S^{*}}}\setminus S\rightarrow[S\setminus{S^{*}}]^{\leq k}$, such that $(S\setminus b(u))\cup\{u\}\in{{\cal F}}$, for all $u\in{{S^{*}}}\setminus S$.*

*We distinguish between two cases, based on the last iteration of the algorithm. *

Case 1:* For all elements $x\in{S^{*}}\setminus S$ the $k$-swap $(x,b(x))$ was rejected because it satisfied $\rho_{(x,b(x))}\leq 0$ (and not because $c_{x}-c(b(x))+c(S)>B$. That is, the swap $(x,b(x))$ did not violate the knapsack constraint.*

*By Case 1 assumption, it follows that no $k$-swap $(x,b(x))$, for an element $x\in{S^{*}}\setminus S$, could increase the value of $f(\cdot)$. Hence, similar to the single matroid case, we have *

[TABLE]

In the next claim, we show that $\sum_{x\in{S^{*}}\setminus S}\left(f(S)-f(S\setminus b(x))\right)\leq k\cdot f(S)$. Thus, in this case, we have that $k$-MatKnap yields a $\frac{1}{k+1}$-approximation to the optimum.

Claim 2**.**

Let $b:{S^{*}}\setminus S\rightarrow[S\setminus{S^{*}}]^{\leq k}$ be the mapping defined in Lemma 3.2. Then $\sum_{x\in{S^{*}}\setminus S}\left(f(S)-f(S\setminus b(x))\right)\leq k\cdot f(S)$.

Proof 3.6**.**

Let $S=\{u_{1},\ldots,u_{|S|}\}$, where the indices are assigned arbitrarily. Fix an element $x\in{S^{*}}\setminus S$, for which $b(x)\neq\emptyset$. Let $b(x)=\{u_{i_{1}},\dots,u_{i_{\ell}}\}$, where $i_{1}<\cdots<i_{\ell}$, and $\ell\leq k$, and let $b^{j}(x)=\{u_{i_{j}},\dots,u_{i_{\ell}}\}$, for $1\leq j\leq\ell$, and $b^{\ell+1}(x)=\emptyset$. We have

[TABLE]

The last inequality follows from submodularity, since for $1\leq j\leq\ell$,

[TABLE]

As each element in $S\setminus{S^{*}}$ appears in at most $k$ subsets $b(x)$, for $x\in{S^{*}}\setminus S$, summing Inequality (3.6) over all elements in ${S^{*}}\setminus S$ we get

[TABLE]

Case 2:* At least one $k$-swap $(x,b(x))$, for $x\in{S^{*}}\setminus S$, considered in the last iteration, violated the knapsack constraint; namely, satisfied $c_{x}-c(b(x))+c(S)>B$.*

As in the single matroid case, denote by $S^{\ell}$ the subset of elements in the solution after iteration $\ell$ of the algorithm, for $\ell\geq 0$, where $S^{0}=Y$. Let $b_{\ell}:{S^{*}}\setminus S^{\ell}\rightarrow[S^{\ell}\setminus{S^{*}}]^{\leq k}$ be the mapping guaranteed by Lemma 3.2. Recall that in all iterations $t$ except the last, the algorithm performs a swap $(x_{t},{\bar{y}}_{t})$, where ${\bar{y}}_{t}=b(x_{t})$. Denote $\rho_{(x_{t},{\bar{y}}_{t})}$ by $\rho_{t}$.

Let $t^{*}+1$ be the first iteration in which a swap $(u_{t^{*}+1},b_{t^{*}}(u_{t^{*}+1}))$, for $u_{t^{*}+1}\in{S^{*}}\setminus S$, was considered and rejected, since it violated the knapsack constraint. By the choice of $t^{*}$, for all $t=0,\ldots,t^{*}-1$, for every $x\in{S^{*}}\setminus{S^{t}}$ such that $(x,b_{t}(x))$ is a swap, we must have $\rho_{(x,b_{t}(x))}\leq\rho_{t+1}$.

*As in the single matroid case, define the function $g(S)=f(S)-f(Y)$.

Using submodularity, we can now write:*

[TABLE]

Following the proofs of claims 1 and 2, we have $\displaystyle{\sum_{x\in{{S^{*}}}\setminus{{S^{t}}}}}(g({{S^{t}}})-g({{S^{t}}}\setminus b_{t}(x)))\leq k\cdot g({{S^{t}}})$. Thus, for any $t=0,\ldots,t^{*}-1$,

[TABLE]

It follows that

[TABLE]

A derivation similar to the one for the single matroid case, combined with Lemma 2.5 and (17), yields

[TABLE]

Finally, using Lemma 3.4,

[TABLE]

Using analogous techniques, as in the case of a single matroid constraint, we can modify the algorithm to obtain the following theorem.

Theorem 3.7**.**

Algorithm $k$-MatKnap can be modified to run in $\tilde{O}(n^{k+5})$ time.

Proof 3.8** (Proof (sketch)).**

Fix some $\varepsilon>0$ (to be set later), and perform a $k$-swap only if the value of $f(\cdot)$ increases at least by a factor of 1+$\frac{\varepsilon}{n^{2}}$ as a result of the swap. As before, the overall number of swaps is bounded by $\frac{n^{2}}{\epsilon}\log n$, and it can be shown that for $\varepsilon\leq\frac{2k+2}{e^{2}-1}$, the approximation ratio remains the same.

We now turn to analyze the running time. We have $O(n^{2})$ guesses of the set $Y$. For each such guess, we have overall $\tilde{O}(n^{2})$ successful $k$-swaps (Note that there are $O(n)$ swaps in which $\emptyset$ is involved.) This implies, as before, an overall of $\tilde{O}(n^{2})$ $k$-swaps. Each $k$-swap requires $O(n^{k+1})$ time. We conclude that the overall running time is $\tilde{O}(n^{k+5})$.

**Acknowledgments: ** We thank Chandra Chekuri and Moran Feldman for helpful comments on an earlier version of the paper.