Typical Approximation Performance for Maximum Coverage Problem

TL;DR

This paper analyzes the typical performance of belief propagation, greedy, and linear programming algorithms for maximum coverage on sparse graphs, revealing thresholds and differences in their typical effectiveness through theoretical and numerical analysis.

Contribution

It provides a detailed analysis of the typical performance thresholds of various approximation algorithms for maximum coverage, highlighting differences not evident in worst-case analysis.

Findings

Belief propagation exhibits two distinct thresholds of replica-symmetry and its breaking.

In low-density regions, belief propagation outperforms other algorithms in typical performance.

Numerical simulations confirm theoretical predictions and reveal relationships among algorithms.

Abstract

This study investigated typical performance of approximation algorithms known as belief propagation, greedy algorithm, and linear-programming relaxation for maximum coverage problems on sparse biregular random graphs. After using the cavity method for a corresponding hard-core lattice--gas model, results show that two distinct thresholds of replica-symmetry and its breaking exist in the typical performance threshold of belief propagation. In the low-density region, the superiority of three algorithms in terms of a typical performance threshold is obtained by some theoretical analyses. Although the greedy algorithm and linear-programming relaxation have the same approximation ratio in worst-case performance, their typical performance thresholds are mutually different, indicating the importance of typical performance. Results of numerical simulations validate the theoretical analyses and imply further mutual relations of approximation algorithms.

Figures

Equations

$$\begin{array}[]{lll}\mbox{Max.}&\displaystyle\sum_{a=1}^{M}y_{a},&\\ \mbox{subject to\ }&\displaystyle y_{a}\leq\sum_{i;\,(i,a)\in E}x_{i}&\forall a\in V_{2},\\ &\displaystyle\sum_{i=1}^{N}x_{i}\leq K,&\\ &x_{i}\in\{0,1\}\>\forall i\in V_{1},&y_{a}\in\{0,1\}\>\forall a\in V_{2}.\end{array}$$

$${\rho }_y({\rho }_x;N)=\frac{1}{M}\overline{\sum _{a=1}^M{y}_a^{\mathrm{opt}}(G,{\rho }_x)},$$

$$\begin{array}[]{lll}\mbox{Max.}&\displaystyle\sum_{a=1}^{M}y_{a},&\\ \mbox{subject to\ }&\displaystyle y_{a}\leq\sum_{i;\,(i,a)\in E}x_{i}&\forall a\in V_{2},\\ &\displaystyle\sum_{i=1}^{N}x_{i}\leq K,&\\ &x_{i}\in[0,1]\>\forall i\in V_{1},&y_{a}\in[0,1]\>\forall a\in V_{2}.\end{array}$$

$${\Xi }_0(\mu ;G)=\sum _{\mathbf{x}\in \{0,1{\}}^N}\sum _{\mathbf{y}\in \{0,1{\}}^M}\exp \left(\mu \sum _{a=1}^M{y}_a\right)H\left(K-\sum _{i=1}^N{x}_i\right)\prod _{a=1}^{M}H\left(\sum _{i\in \partial a}{x}_i-y_a\right),$$

$$\Xi (\mu ;G)=\sum _{\mathbf{x}\in \{0,1{\}}^N}\sum _{\mathbf{y}\in \{0,1{\}}^M}\exp \left({\mu }^{\prime }\sum _{i=1}^N{x}_i+\mu \sum _{a=1}^M{y}_a\right)\prod _{a=1}^{M}H\left(\sum _{i\in \partial a}{x}_i-y_a\right),$$

$$\kappa =-\frac{{\mu }^{\prime }}{\mu }.$$

$${\rho }_x(\mu ,\kappa )={\rho }_x,{\rho }_x(\mu ,\kappa )\equiv {⟨\frac{1}{N}\sum _{i=1}^N{x}_i⟩}_{\mu },$$

$$P_i(x)\simeq Z_i^{-1}{e}^{-\mu \kappa x}\prod _{a\in \partial i}{P}_{a\to i}(x),,$$

$$P_a(y)\simeq Z_a^{-1}{e}^{\mu y}\sum _{{\mathbf{x}}_{\partial a}}H\left(\sum _{i\in \partial a}{x}_i-y\right)\prod _{i\in \partial a}{P}_{i\to a}(x),$$

$$P_{i\to a}(x)$$

$$P_{a\to i}(x)$$

$$h_{ia}$$

$${\hat{h}}_{ai}$$

$${\xi }_i$$

$${\hat{\xi }}_a$$

$$h$$

$$\hat{h}$$

$$\xi$$

$$\hat{\xi }$$

$$\frac{e^{\mu \xi }}{1+e^{\mu \xi }}={\rho }_x.$$

$$\kappa =L\hat{h}-\frac{1}{\mu }\ln r,$$

$${\left(1+rx\right)}^{R-1}(1-x)=\frac{1}{1+e^{-\mu }}.$$

$$x=\frac{e^{-\mu }}{1-(R-1)r}+O(e^{-2\mu }).$$

$$\underset{M\to \infty }{\lim }\overline{{⟨\frac{1}{M}\sum _{i=1}^M{y}_a⟩}_{\mu }}$$

$$=R{\rho }_x+O(e^{-\mu })$$

$${\rho }_y^{\mathrm{RS}}=R{\rho }_x.$$

$${\chi }_{\mathrm{SG}}(\mu )=\frac{1}{N+M}\overline{\sum _{i\in V_1}\sum _{a\in V_2}(⟨x_i{y}_a{⟩}_{\mu }-⟨x_i{⟩}_{\mu }⟨y_a{⟩}_{\mu }{)}^2}.$$

$${\chi }_{\mathrm{SG}}(\mu )\simeq \sum _{d=0}^{\infty }{\lambda }^d,\lambda =\mathbb{E}\left[\sum _{j\in \partial a\backslash i;b\in \partial j\backslash a}{\left(\frac{\partial {\hat{h}}_{ai}}{\partial {\hat{h}}_{bj}}\right)}^2\right],$$

$$\lambda$$

$$\to (L-1)(R-1)r^2(\mu \to \infty ).$$

$${\rho }_x^{\mathrm{RS}}=\frac{1}{1+\sqrt{(L-1)(R-1)}}.$$

$$(L-1)(R-1){\left[\frac{{\rho }_x^{{\mathrm{RS}}^{\prime }}(1-x^{\ast })}{(1-{\rho }_x^{{\mathrm{RS}}^{\prime }})(1+x^{\ast })}\right]}^2=1,$$

$${\left(1+\frac{{\rho }_x^{{\mathrm{RS}}^{\prime }}}{1-{\rho }_x^{{\mathrm{RS}}^{\prime }}}{x}^{\ast }\right)}^{R-1}(1-x^{\ast })=1.$$

$$NV(T+1)=NV(T)-1-L(R-1)\frac{NV(T)}{N-RT}+O(N^{-1}),$$

Full text

I Introduction

Approximation algorithms, which are important for hard optimization problems, have attracted researchers’ interest because, for NP-hard optimization problems, one encounters difficulty when solving optimal solutions exactly in polynomial time of the problem size. Since the P versus NP problem arose, development and performance analyses of approximation algorithms have persisted as a central issue of computer science and operations research.

Two performance evaluations of approximation algorithms exist: worst-case performance and typical (or average-case) performance. As described in this paper, we specifically examine the latter mainly using statistical–mechanical methods, although the former has been investigated mainly in the literature of theoretical computer science Vazirani (2003). Worst-case performance is defined by the pair of an optimization problem and its approximation algorithm. An approximation ratio is then defined by the maximal ratio of an optimal value to an approximation value over all instances if the problem is a maximization problem (and vice versa, otherwise). It is important to provide strict performance guarantee of the algorithm, although the worst-case instance is sometimes pathological. However, the typical performance is defined as the average performance of approximation algorithm for a given optimization problem over randomized instances. It is sometimes useful for practical use. It sheds light on properties of optimization problems and approximation algorithms in a perspective that is different from worst-case analysis.

An interesting point of the typical property of optimization problems is found in a close relation to the spin-glass theory in statistical physics. Extensive studies of various computational problems have revealed that the concept of replica symmetry (RS) and its breaking (RSB) in spin-glass theory reflects average computational complexity Monasson et al. (1999) and structure of the solution space Mézard et al. (2003) of the problems.

Typical performance of approximation algorithms often exhibits a phase transition on typical goodness or accuracy of approximation, which is called threshold phenomena in the literature of theoretical computer science Karp and Sipser (1981). Some approximation algorithms for minimum vertex covers (min-VC) are good examples of threshold phenomena. The problem is defined on a graph. The randomized problem is characterized using a random graph ensemble with $c$ being the average degree. From a statistical–mechanical perspective, the problem on Erdös-Rényi random graphs has the RS-RSB threshold at $c=e=2.71\dots$ Weigt and Hartmann (2000). Moreover, three approximation algorithms have been investigated: belief propagation (BP) (or message passing) Weigt and Zhou (2006), greedy leaf-removal algorithm Bauer and Golinelli (2001), and linear-programming (LP) relaxation Dewenter and Hartmann (2012); Takabe and Hukushima (2014a). Approximation algorithms other than BP naively have no direct connection to the spin-glass theory. Nevertheless, for Erdös-Rényi random graphs, these algorithms have the same performance threshold as the RS-RSB threshold. However, subsequent studies of general random graphs Takabe and Hukushima (2016a) indicate that their thresholds are not equivalent for some random graphs because of their graph structure. Using statistical–mechanical techniques, typical performance has been studied for variants of leaf removal Liu et al. (2012); Takabe and Hukushima (2014b); Lucibello and Ricci-Tersenghi (2014); Habibulla et al. (2015); Takahashi et al. (2017) and relaxation technique such as LP relaxation Schawe and Hartmann (2016); Takabe and Hukushima (2016b) and semidefinite-programming relaxation Javanmard et al. (2016). These studies have revealed not only typical approximate performance itself but also a suggestive connection to typical properties of optimization problems, random graph structure, and the spin-glass theory.

As described in this paper, we examine the unweighted maximum coverage (max-COV) problem defined in the next section. Although the max-COV belongs to the class of NP-hard, it has several practical applications such as pan and scan problems Johnson and Bar-Noy (2011), multi-topic blog watch Saha and Getoor (2009), and text summarization Takamura and Okumura (2009). Our main purpose is to examine the typical performance of approximation algorithms for the problem and to compare their performance thresholds. We specifically examine three approximation algorithms: belief propagation, greedy algorithm, and LP relaxation. Some statistical–mechanical methods applied to both analytical and numerical analyses together with mathematical rigorous discussions clarify typical performance of those algorithms and a suggestive mutual relation among approximation algorithms.

This paper is organized as follows. In the next section, we define details of the max-COV and its approximation algorithms. As a random graph ensemble, biregular random graphs are also defined. In Section III, a hard-core lattice–gas model for the problem is introduced. Its BP equations are obtained based on Bethe–Peierls approximation. In Section IV, we present a study of the typical performance of BP using the RS cavity method. Calculation of the spin-glass susceptibility provides the threshold below which BP typically approximate max-COV with high accuracy. In Section V, the greedy algorithm is analyzed based on a mean-field rate equation of its deletion process. Using the obtained solution, the typical performance threshold of the greedy algorithm is evaluated. In Section VI, LP-relaxed approximate values are evaluated rigorously. The theorem is proved using the weak duality theorem. In Section VII, we describe some numerical results which support the validity of theoretical analyses presented in the previous sections. We also execute some additional simulations to consider the typical performance of a modified greedy algorithm and randomized rounding of LP relaxation. These results provide suggestive relation between approximation algorithms. The last section is devoted to a summary and discussion of the results.

II Max-Cover problem and approximation algorithms

II.1 Maximum coverage problem

The max-COV is defined as follows: Let $S$ be a set of $M$ elements and $\mathcal{S}=\{S_{1},\cdots,S_{N}\}$ be a collection of subsets of $S$. For a given positive integer $K(\leq N)$, the problem is to choose at most $K$ subsets to maximize the total number of elements in the union of chosen subsets. An example of an instance with $M=4$ and $N=3$ is shown at the left-hand side of Fig. 1. Given that $K=2$, one can select subsets $S_{1}$ and $S_{3}$ such that all elements are included in the union of the subsets.

Converting an instance to a bipartite graph, one obtains an integer-programming (IP) representation of the max-COV. Let $G=(V_{1},V_{2},E)$ be a bipartite graph where each edge in $E$ connects to a vertex in $V_{1}$ and a vertex in $V_{2}$. Vertices in $V_{1}$ and $V_{2}$ are labeled respectively by $i\in\{1,\dots,N\}$ and $a\in\{1,\dots,M\}$. $V_{1}$ corresponds to a collection of subsets $\mathcal{S}$, whereas $V_{2}$ represents $S$. Each edge $(i,a)$ is set if subset $S_{i}$ includes element $a$. For Fig. 1, the left example is converted to a bipartite graph in the right-hand side. We then introduce binary variables $\{x_{i}\}$ and $\{y_{a}\}$, respectively, to $V_{1}$ and $V_{2}$. Also, $x_{i}$ is set to one if vertex $i$ (or subset $S_{i}$) is selected and zero otherwise. Similarly, $y_{a}$ is set to one if vertex $a$ is connected to a selected vertex (or element $a$ belongs to the union of chosen subsets) and zero otherwise. The problem is therefore represented by the following IP problem:

[TABLE]

The inequality in the second constraint can be replaced with an equality sign because the problem is a maximization problem.

As described in this paper, typical-case property of the max-COV is analyzed by randomizing its instance. Then, we introduce $\rho_{x}=K/N$ and assume that $\rho_{x}\in[0,1]$ is constant. For random bipartite graphs, we specifically examine $(L,R)$-biregular random graphs where degrees of each vertex in $V_{1}$ and $V_{2}$ respectively denote $L$ and $R$. Using this simple random graph ensemble, our statistical–mechanical analyses reveal interesting properties of the problem.

To take the large graph limit, the number of vertices in $V_{2}$ is rescaled as $M=\alpha N$ with a constant factor $\alpha$. We assume that a random graph is sparse, i.e., $L$ and $R$ are constant with respect to $N$. For the randomized max-COV, we define an average optimal cover ratio over random graphs with cardinality $N+M$ and a given $\rho_{x}$ by

[TABLE]

where $\{y_{a}^{\mathrm{opt}}(G,\rho_{x})\}$ represents optimal solutions in $V_{2}$ and $\overline{(\cdots)}$ represents an average over random bipartite graphs with size $N+M$. Its limiting value to $N\rightarrow\infty$ is denoted by $\rho_{y}$. We simply call it the average optimal cover ratio.

II.2 Approximation algorithms

We introduce three approximation methods for max-COVs. The first algorithm is belief propagation (BP). The recursive equations called BP equations are derived from Bethe–Peierls approximation for a spin system corresponding to a given optimization problem. For systems on trees, graphs with no cycles, the Bethe–Peierls approximation and BP are exact. In general, however, there exist cycles in a graph yielding correlations between variables or spins, which results in inexact estimation of solutions (or configurations). BP on graphs with some cycles is regarded as an approximation method. It is called loopy BP Mézard and Montanari (2009).

The second one is a simple greedy algorithm. At each step, this algorithm has the following procedure: (i) choose one vertex named $i$ with the maximum degree in $V_{1}$, (ii) delete vertices neighboring to vertex $i$ from $V_{2}$, and (iii) update $V_{1}$ to $V_{1}\backslash i$ and return to (i) if $|V_{1}|>N-K$. This simple algorithm gives an approximation ratio of $1-1/e$ Nemhauser et al. (1978). The problem cannot be approximated within this ratio unless P$=$NP Feige (1998).

The last is linear programming (LP) relaxation. An integer programming problem including the max-COV is relaxed to LP problems by replacing integral constraints with real constraints. The LP-relaxed max-COV is given as

[TABLE]

Actually, LP approximation value gives the upper bound of the problem. Because LP problems are solvable exactly in polynomial time, LP relaxation is a widely used approximation technique. The approximate solution obtained by LP relaxation usually involves non-integers. One must round those numbers appropriately to obtain an approximate integral solution for the IP problem. Here, we consider randomized rounding Raghavan and Tompson (1987). Using the obtained approximation solution $\{x_{i}^{\mathrm{LP}}\}$, one selects vertex $i\in V_{1}$ to set $I\subset V_{1}$ with probability $x_{i}^{\mathrm{LP}}/K$ up to $K$ vertices. Then, the rounded solution $\{x_{i}^{\mathrm{LPr}}\}$ is set to $x_{i}^{\mathrm{LPr}}=1$ for $i\in I$ and $x_{i}^{\mathrm{LPr}}=0$ otherwise. The rounded approximation value is readily calculated from $\{x_{i}^{\mathrm{LPr}}\}$. In terms of the worst-case performance, LP relaxation and its randomized rounding in expectation has the same approximation ratio as the greedy algorithm.

As described in this paper, we study the typical performance of these approximation algorithms. It is evaluated by approximate values averaged over randomized max-COVs defined in the last subsection. Similar to the average optimal cover ratio, the average cover ratio is defined as the average ratio of the approximate value to the cardinality $M$ of vertex set $V_{2}$. It is regarded as exhibiting good typical performance if the average cover ratio obtained by an approximation algorithm is equal to the average optimal cover ratio in the large-$N$ limit. The main aim of this paper is to evaluate the typical performance threshold of $\rho_{x}$ below which an approximate algorithm exhibits good typical performance with $L$ and $R$ fixed. Evaluation of the approximation algorithms is accomplished by comparing their typical performance thresholds.

III BP equations for Max-COV

As explained in this section, BP equations for max-COV are derived based on the statistical–mechanical model for the problem. We set particles on vertices in $V_{1}$ and $V_{2}$, which respectively occupy vertex $i$ and $a$ if $x_{i}=1$ and $y_{a}=1$. The hard-core lattice-gas model for max-COVs on bipartite graph $G$ is then naively given as the following partition function.

[TABLE]

Therein, $H(x)=1\,(x\geq 0),\,0\,(x<0)$ is the Heaviside step function and $\partial a=\{i\in V_{1}\mid(i,a)\in E\}$ stands for a set of neighbors of vertex $a\in V_{2}$. In the partition function, $\mu$ represents a chemical potential for particles on $V_{2}$. One can construct BP equations for this model. However, it is inconvenient for practical use because of the constraint $\sum_{i=1}^{N}x_{i}\leq K$.

Therefore, we use the following alternative partition function of

[TABLE]

where $\mu^{\prime}=\mu^{\prime}(\mu,\rho_{x};G)$ is a chemical potential for particles on $V_{1}$. The ratio of $\mu^{\prime}$ with respect to $\mu$ is defined as

[TABLE]

This parameter is regarded as a Lagrange multiplier for the constraint on the number of selected vertices. The appropriate value of $\kappa$ must satisfy the condition given by

[TABLE]

where $\langle\dots\rangle_{\mu}$ is a grand-canonical average with a given $\mu$. To consider ground states, we take the large-$\mu$ limit with parameter $\kappa$ fixed.

First, we construct BP equations for Eq. (5). Using the Bethe–Peierls approximation, the single spin probability $P_{i}(x)$ which $x_{i}$ takes $x$ is

[TABLE]

where $\partial i=\{a\in V_{2}\mid(i,a)\in E\}$ is a set of neighbors of vertex $i\in V_{1}$. $Z_{\ast}$ is a normalization factor hereinafter. $P_{a\rightarrow i}(x)$ represents the marginal probability of $\bm{x}_{\partial a\backslash i}$ and $y_{a}$ under the condition $x_{i}=x$. Similarly, single spin probability $P_{a}(y)$ that $y_{a}$ takes $y$ reads

[TABLE]

where $P_{i\rightarrow a}(x)$ is the probability of $x_{i}$ taking $x$ on the cavity graph $G\backslash a$. These probabilities satisfy the following recursive relations under the same approximation.

[TABLE]

To take the large-$\mu$ limit later, it is convenient to introduce cavity fields $\{h_{ia}\}$ and $\{\hat{h}_{ai}\}$ defined respectively by $P_{i\rightarrow a}(x)\propto\exp(\mu h_{ia}x)$ and $P_{a\rightarrow i}(y)\propto\exp(\mu\hat{h}_{ai}y)$. Then, BP equations for cavity fields are given as

[TABLE]

By rescaling the single spin probability as $P_{i}(x)\propto\exp(\mu\xi_{i}x)$ and $P_{a}(y)\propto\exp(\mu\hat{\xi}_{a}y)$ using local fields $\{\xi_{i}\}$ and $\{\hat{\xi}_{a}\}$, Eqs. (8) and (9) then read

[TABLE]

One can estimate the single spin probability by solving BP equations (12) and (13) as the loopy belief propagation.

IV Replica-Symmetric Solution

In this section, typical performance of BP is studied using the RS cavity method based on the simplest RS ansatz. The RS ansatz assumes that cavity fields $\{h\}$ and $\{\hat{h}\}$ are independent random variables respectively following probability distributions $P(h)$ and $\hat{P}(\hat{h})$. For biregular random graphs, it is apparent that these distributions have no variance because of the absence of fluctuation of degree in $V_{1}$ and $V_{2}$. We therefore introduce the cavity fields $h$ and $\hat{h}$ on $(L,R)$-biregular random graphs. Using BP equations (12) and (13), they satisfy the following RS cavity equations in the large-$N$ limit as

[TABLE]

Similarly, the local fields on $(L,R)$-biregular random graphs are set to $\xi$ and $\hat{\xi}$, which satisfy

[TABLE]

According to Eq. (7), for a given $\rho_{x}$, the local field $\xi$ is determined as

[TABLE]

Then, using Eqs. (18) and (20), the appropriate parameter $\kappa$ is represented as

[TABLE]

where $r=\rho_{x}/(1-\rho_{x})$.

By substituting Eqs. (16) and (21) to Eq. (17) and by introducing $x=e^{-\mu\hat{h}}$, one obtains the self-consistent equation as

[TABLE]

The order of this solution changes depending on the value of $\rho_{x}$. The solution $x$ vanishes as $\mu$ becomes large if $\rho_{x}<1/R$ holds. Using Taylor expansion of the left-hand side of Eq. (22), it is estimated as

[TABLE]

Using Eq. (19), an average density of particles on $V_{2}$ reads

[TABLE]

Taking the large-$\mu$ limit, the average cover ratio for $\rho_{x}<1/R$ is obtained as

[TABLE]

This relation indicates that each vertex in $V_{2}$ is connected to at most one chosen vertex in $V_{1}$. However, if $\rho_{x}$ is larger than $1/R$ ($R\geq 2$), then the solution $x$ remains constant for sufficiently large $\mu$. In this case, a simple solution $\rho_{y}^{\mathrm{RS}}=1$ is obtained by Eq. (24) because the average cover ratio in Eq. (26) touches to one at $\rho_{x}=1/R$.

Before closing this section, we discuss the stability of the RS solutions using spin-glass susceptibility defined as

[TABLE]

Another representation of $\chi_{\rm SG}$ using cavity fields is known based on the fluctuation–dissipation theorem Rivoire et al. (2003). As shown in Mézard and Tarzia (2007), in the case of biregular random graphs, it reads

[TABLE]

where $\mathbb{E}[\cdots]$ represents an average over vertices in $V_{1}$ and random graphs. The cavity fields are mutually correlated in general. Actually, BP cannot converge any more Zdeborová (2009) if the susceptibility diverges. In this sense, the divergence of $\chi_{\rm SG}$ not only means the instability of the RS solution, but also poor typical performance of BP as an approximation algorithm in the static sense. In the low-density region where $\rho_{x}<1/R$ holds, it is evaluated as

[TABLE]

The RS-RSB threshold in the large-$\mu$ limit therefore reads

[TABLE]

Otherwise, the other threshold $\rho_{x}^{\mathrm{RS}^{\prime}}(>1/R)$ satisfies

[TABLE]

where $x^{\ast}$ is a solution of

[TABLE]

In Fig. 2, RS-RSB thresholds $\rho_{x}^{\mathrm{RS}}$ and $\rho_{x}^{\mathrm{RS}^{\prime}}$ on $(3R,R)$-regular random graphs are shown as a function of $R$. Except for the case $R=1$, the RS-RSB thresholds separate RS and RSB regions. The RSB region remains for a finite $\rho_{x}$ while they converge to zero in the large-$R$ limit.

V Typical Analysis of Greedy Algorithm

In this section, typical behavior of the greedy algorithm on $(L,R)$-biregular random graphs is investigated in the low-density region of $\rho_{x}<1/R$. The corresponding RS solution indicates that one can choose vertices in $V_{1}$ without overlapping their neighbors. It is therefore sufficient to analyze the fraction of vertices in $V_{1}$ with maximum degree during the deletion process. Here we analyze a rate equation that is used frequently for analyses of similar greedy algorithms Karp and Sipser (1981); Weigt (2002). Let $V(T)$ be the expected number of vertices with maximum degree in $V_{1}$ at $T$-th step of the algorithm. By the definition of the algorithm in sec. II.2, we find that

[TABLE]

where the assumption $L=O(1)$ is used. By introducing $v(t)=V(T)/N$ and $t=T/N$, we obtain the following differential equation in the large graph limit.

[TABLE]

Under the initial condition $v(0)=1$, the solution reads

[TABLE]

Let $\rho_{x}^{\mathrm{g}}$ be a threshold below which vertices with the maximum degree are left at the end of the algorithm. Considering that $t$ represents a fraction of chosen vertices in $V_{1}$, we find $v(\rho_{x}^{\mathrm{g}})=0$, that is,

[TABLE]

If $\rho_{x}<\rho_{x}^{\mathrm{g}}$, then no chosen vertex in $V_{1}$ has overlapped neighbors in $V_{2}$ resulting in good typical performance of the algorithm, i.e., $\rho_{y}^{\mathrm{g}}=R\rho_{x}$. However, the algorithm typically mistakes selections of vertices in $V_{1}$ and underestimates the average cover ratio. Results show that $\rho_{x}^{\mathrm{g}}\leq\rho_{x}^{\mathrm{RS}}$ where the equality holds for $R=1$, indicating that BP is better than the greedy algorithm in terms of the typical performance threshold.

VI Analysis of LP relaxation

An LP-relaxed value of the max-COV on any biregular graph is evaluated exactly using the LP duality Rockafellar (1972) as follows.

Theorem 1

An LP-relaxed value of the max-COV on any $(L,R)$-biregular graph is $LK$ if $K\leq 1/R$ holds.

Proof.

$x_{i}=K/N$ ($1\leq i\leq N$), $y_{a}=RK/N$ ($1\leq a\leq M$) is a feasible solution of LP-relaxed max-COV (3). The value of the cost function is then $RKM/N=LK$.

The Lagrangian function of Eq. (3) is written as

[TABLE]

where $p\in\mathbb{R}$ and $\bm{q}\in\mathbb{R}^{M}$. Using the weak duality theorem, one finds the following inequalities.

[TABLE]

Consequently, $\max_{\bm{x},\bm{y}}L(\bm{x},\bm{y},{p},\bm{q})$ is an upper bound of the LP-relaxed value. Because Eq. (37) is represented by

[TABLE]

where a solution $x_{i}=y_{a}=1$ ($1\leq i\leq N,1\leq a\leq M$) realizes the maximum of the function with $p,\bm{q}$ satisfying $-p+\sum_{a\in\partial i}q_{a}\geq 0$ and $1-q_{a}\geq 0(1\leq a\leq M)$. Assuming that $q_{a}=q$ holds for $1\leq a\leq M$, one finds that

[TABLE]

On the right-hand side, setting $(p,q)=(M/N,K/N)$ results in $LK$, which indicates that the upper bound of the LP-relaxed max-COV on $(L,R)$-biregular graph is $LK$. Because the approximate solution denoted above achieves this bound, the LP-relaxed value is equivalent to the bound. $\square$ $\square$

This theorem claims that $\rho_{y}=R\rho_{x}$ ($\rho_{x}\leq 1/R$), suggesting that LP relaxation typically finds good approximate values in the RS regime. The LP-relaxed value is equivalent to $\rho_{y}^{\mathrm{RS}}$ in the high-density regime where $\rho_{x}>1/R$.

VII Numerical results

This section presents description of some numerical results for validation of the theoretical analyses. Here, we set $(L,R)=(9,3)$ as an example. Then, the RS-RSB threshold and the typical performance threshold of the greedy algorithm are evaluated as $\rho_{x}^{\mathrm{RS}}=0.2$ and $\rho_{x}^{\mathrm{g}}=(1-2^{-4/5})/3=0.1418\dots$. Biregular random graphs are generated based on implementation of the configuration model Newman et al. (2001). At least $400$ random graphs are used to take a random graph average.

VII.1 Average cover ratio

To examine the validity of theoretical analyses on the typical performance of approximation algorithms, the average cover ratio $\rho_{y}$ is evaluated using several methods.

We employ a BP-guided decimation (BPD) algorithm Mézard and Montanari (2009) as a variant of loopy BP to obtain a feasible solution satisfying all the constraints. The algorithm fixes a value of a variable based on a solution of BP equations (11) until all variables are fixed to either zero or one. When a variable is fixed, BP equations are updated by applying the fixed value. If the number of variables in $V_{1}$ fixed to one reaches to $K$, the remainders of $\bm{x}$ are immediately fixed to zero to satisfy the constraint. An approximate value of the algorithm is thus a function of the parameter $\kappa$, which depends on inputs, i.e., $\mu$, $K$ and a graph. Here, the BP equations with $\mu=20$ are solved iteratively up to $150$ steps, which enables the algorithm to fix a variable practically even if the iterations cannot reach to the RS fixed point. The parameter $\kappa$ is tuned to maximize an approximate value while the detail will be reported elsewhere. In addition, the RS estimation of average cover ratio $\rho_{y}^{\mathrm{RS}}$ in the RS regime $\rho_{x}<\rho_{x}^{\mathrm{RS}}$ is used for comparison to BPD though its typical performance is possibly evaluated directly as in Ricci-Tersenghi and Semerjian (2009).

The theoretical result presented in section VI is used for LP relaxation, which is valid for arbitrary biregular graphs. Average approximate values of the greedy algorithm are estimated numerically with $N=10^{3}$.

We also evaluate the average optimal value using the replica exchange Monte Carlo (EMC) method Hukushima and Nemoto (1996). Results show that single-spin flip updates of the model (4) takes a long relaxation time for equilibration even using the exchange MC, indicating the existence of deep valleys of the free energy. Consequently, it is necessary to accelerate equilibration for a system with sufficiently small chemical potential $\mu$. To avoid such slow relaxation, we consider an alternative lattice–gas model on bipartite graph $G$ represented as

[TABLE]

where $\theta(x)$ takes one if $x>0$ and zero otherwise, and $\delta(x,y)$ is Kronecker’s delta. In this model, variables $\bm{y}$ are eliminated because $y_{a}=\theta(\sum_{i\in\partial a}x_{i})$ holds in the large-$\mu$ limit. The ground states of the alternative model therefore correspond to the optimal solutions of the max-COV on the same graph. Moreover, single-spin flip updates in the model are substantially equivalent to multi-spin updates in the original model (4), which makes the relaxation time to equilibrium states markedly short. Because optimal values are invariant by replacing inequality constraint $\sum_{i=1}^{N}x_{i}\leq K$ to equality one, the equality constraint is adopted and Kawasaki dynamics for a density-conserved system Kawasaki (1966) is applied.

The results are presented in Fig. 3. Numerical results are compatible to the RS estimation nearly up to the RS-RSB threshold $\rho_{x}^{\mathrm{RS}}=0.2$ for the BPD and EMC, and up to the typical performance threshold $\rho_{x}^{\mathrm{g}}$ predicted analytically for the greedy algorithm. As shown in the inset of Fig. 3, the average cover ratio by the greedy algorithm deviates from the average optimal ratio above $\rho_{x}^{\mathrm{g}}$, while other estimates split above $\rho_{x}^{\mathrm{RS}}$, indicating that the Bethe–Peierls approximation and LP relaxation are no longer appropriate. In fact, as shown in Fig. 4, the loopy BP for $N=400$ fails its convergence above $\rho_{x}\simeq 0.195$, which may imply the existence of dynamical one-step RSB phase. It is expected that BPD finds approximate solutions with good accuracy if a corresponding loopy BP converges to a fixed point. Otherwise, decimation in BPD is performed based on a wrong estimation by the Bethe–Peierls approximation. We emphasize, however, that BPD finds relatively good approximate solutions even in the RSB phase as shown in Fig. 3. From these observations, we confirm that statistical–mechanical analyses successfully predict typical performance thresholds by approximation algorithms.

VII.2 Greedy algorithm and its variant

Here we examine the greedy algorithm more closely. To validate the correctness of our analysis, we specifically examine a fraction of selected vertices without the maximum degree. The value $r_{\mathrm{g}}$ is evaluated by

[TABLE]

where $\rho_{x}^{\mathrm{g}}$ is given as Eq. (36). The algorithm finds an optimal solution of the problem if $r_{\mathrm{g}}=0$. As depicted in Fig. 5, the analytical estimations of $r_{\mathrm{g}}$ averaged over biregular random graphs agree well with the numerical results. The fraction arises at $\rho_{x}^{\mathrm{g}}$ corresponding to the typical performance threshold.

Additionally, we modify the greedy algorithm to reduce the gap of its typical performance to that of BP. As suggested above, the reason lies in the point that the greedy algorithm often chooses wrong vertices leading to shortage of vertices with maximum degree. To avoid the situation, one must select a vertex to preserve as many vertices to a maximum degree as possible, meaning that the optimal selection requires an exhaustive search. We therefore propose a modified algorithm in consideration of the influence of the selection on other vertices in $V_{1}$, the simplest improved algorithm toward the optimal selection. Let $\partial^{2}i=\{j\in V_{1}\backslash i\mid j\in\partial a\,(\forall a\in\partial i)\}$ be a set of the second neighbors of vertex $i\in V_{1}$. We also define the subset of vertices with the maximum degree in $V_{1}$ by $W_{1}$. The modified greedy algorithm is given as follows: at each step, (i) choose vertex named $i\in W_{1}$ so that $|\partial^{2}i\cap W_{1}|$ is minimized, (ii) delete vertices neighboring to vertex $i$ from $V_{2}$, and (iii) update $W_{1}$ and $V_{1}$ and return to (i) if $|V_{1}|>N-K$. In Fig. 5, the fraction $r_{\mathrm{g}}$ of the modified algorithm is also shown with that of the original greedy algorithm. As shown in Fig. 5, the typical performance threshold of the proposed algorithm is improved, although it is still below the RS-RSB threshold. This fact suggests that the typical performance of greedy algorithms depends of approximate choices and BP selects vertices better than those greedy selections.

VII.3 Randomized rounding of LP relaxation

In Sec. VII.1, we examine the typical performance of LP relaxation in terms of its approximate value. In this subsection, randomized rounding, a practical means of constructing a feasible integer solution from LP relaxation, is applied to LP-relaxed solutions. It is noteworthy that the typical performance of randomized rounding probably depends on the selection of an LP solver and its setting. Here, we use IBM ILOG CPLEX with a default setting. The approximate value of the rounded solution is compared to that of the RS solution in the RS phase, which is $LK$. One considers that the rounding finds an optimal integer solution of the problem if two estimates mutually coincide. We define a success ratio $p_{r}$ by the fraction of random graphs on which the rounding finds an optimal solution. Fig. 6 presents the average approximation ratio $\rho_{y}^{r}$ and the success ratio $p_{r}$ as a function of $\rho_{x}$, which strongly suggests that the randomized rounding exhibits a phase transition in terms of its typical performance. The threshold $\rho_{x}^{\mathrm{r}}$ is less than $0.1$, which is much lower than $\rho_{x}^{\mathrm{g}}$, whereas the LP-relaxed value is regarded as nearly optimal in the RS region $\rho_{x}<\rho_{x}^{\mathrm{RS}}=0.2$. We therefore conclude that the typical performance of the randomized rounding of LP-relaxed solution is inferior to that of the greedy algorithm.

VIII Summary and Discussion

As described in this paper, we investigate the typical performance of approximation algorithms called belief propagation, greedy algorithm, and linear-programming relaxation for maximum coverage problem on sparse biregular random graphs. The typical performance of BP is studied by application of the RS cavity method to a correspondent hard-core lattice–gas model. Results show that, in the large-$\mu$ limit, there exist two distinct RS-RSB thresholds regarded as typical performance thresholds of BP. In addition, the greedy algorithm performance and LP relaxation were studied especially in the low-density region. Results show that the typical performance threshold of the greedy algorithm is lower than that of BP and that LP-relaxed values are always equivalent to the RS solutions leading to the threshold equivalent to that of BP. Those analytical results were validated by executing some numerical simulations. Results of additional numerical studies suggest that BP typically works better than the modified greedy algorithm and that randomized rounding of LP-relaxed solutions has a lower threshold than the greedy algorithm.

To assess the typical performance of BP as an approximation algorithm, we concentrated on statistical–mechanical analysis of max-COV up to the RS level. Further analyses based on the one step RSB will be necessary to reveal statistical–mechanical properties of the problem and typical performance analysis of another algorithm called survey propagation. Another possible avenue of future work is the extension of our analyses to other random bipartite graphs. As with min-VCs, it is an attractive question whether the magnitude relation of typical performance thresholds changes depending on the random graph ensembles.

Our results provide not only respective typical performance of approximation algorithms but also their suggestive mutual relations. Specifically examining LP relaxation, it is worth emphasizing that the LP relaxation finds good approximate values compared to optimal values but the typical performance of its randomized rounding has the smallest threshold among approximation algorithms studied here. To fill the gap of thresholds, it is important to examine modifications of LP relaxation such as the cutting-plane approach Schrijver (1998). As for the greedy algorithms and their modification, numerical results suggest that evaluation of the influence of their deletion process affects the marked improvement of the typical performance threshold. The fact that BP is better than the greedy algorithm and its modification indicates that BP incorporates the influence more efficiently. These suggestions are expected to be of great help to understand properties and relations of approximation algorithms in terms of typical performance. In addition, our analyses of the greedy algorithm and randomized rounding of LP relaxation illustrate that typical case and worst case evaluations capture different notions of approximate performance in optimization problems. This fact indicates the importance of the typical-case analysis of approximation algorithms. We hope that the arguments and results presented herein stimulate further studies and that the typical performance analyses of approximation algorithms will attract the interest of researchers in many diverse fields.

Acknowledgements.

ST warmly thanks T. Takaguchi for stimulating this study. The use of IBM ILOG CPLEX has been supported by the IBM Academic Initiative. This research was supported by JSPS KAKENHI Grant Nos. 25120010 (KH), 16K16011 (TM), and 15J09001 (ST).