Combinatorial persistency criteria for multicut and max-cut

TL;DR

This paper introduces new criteria and algorithms for identifying partial optimal solutions in multicut and max-cut problems, enhancing preprocessing and solution quality in various applications.

Contribution

It develops novel persistency criteria and efficient algorithms for verifying partial optimality in multicut and max-cut problems, applicable in practical scenarios.

Findings

Effective in reducing problem sizes through preprocessing.

Able to compute partial optimality guarantees for heuristic solutions.

Demonstrated success on real-world instances from multiple domains.

Abstract

In combinatorial optimization, partial variable assignments are called persistent if they agree with some optimal solution. We propose persistency criteria for the multicut and max-cut problem as well as fast combinatorial routines to verify them. The criteria that we derive are based on mappings that improve feasible multicuts, respectively cuts. Our elementary criteria can be checked enumeratively. The more advanced ones rely on fast algorithms for upper and lower bounds for the respective cut problems and max-flow techniques for auxiliary min-cut problems. Our methods can be used as a preprocessing technique for reducing problem sizes or for computing partial optimality guarantees for solutions output by heuristic solvers. We show the efficacy of our methods on instances of both problems from computer vision, biomedical image analysis and statistical physics.

Tables

Table 1. Table 1: The table gathers for each data set the number of instances ( # ​ I # 𝐼 \#I ), the graph sizes and instance type ( P ).

Data set	$\#I$	$\left|V\right|$	$\left|E\right|$	P
Image Seg.	100	156–3764	439–10970	PMC
Knott-3D-150	8	572–972	3381–5656	PMC
Knott-3D-300	8	3846–5896	23k–36k	PMC
Knott-3D-450	8	15k–17k	94k–107k	PMC
Knott-3D-550	8	27k–31k	173k–195k	PMC
Mod. Clustering	6	34–115	561–6555	PMC
CREMI-small	3	20k–35k	170k–235k	PMC
CREMI-large	3	430k–620k	3.2m–4.1m	PMC
Fruit-Fly Level 1–4	4	5m–11m	28m–72m	PMC
Fruit-Fly Global	1	90m	650m	PMC
Ising Chain	30	100–300	4950–44850	PCUT
2D Torus	9	100–400	200–800	PCUT
3D Torus	9	125–343	375–1029	PCUT
Deconvolution	2	1001	11k–34k	PCUT
Super Resolution	2	5247	15k–25k	PCUT
Texture Restoration	4	7k–22k	59k–195k	PCUT

Table 2. Table 2: For each dataset the table reports the average fraction of remaining nodes and edges after applying our method, respectively the method from [ 29 ] (lower is better). † Results for Fruit-Fly Global are without ICP-based candidate subgraphs.

	Our		[29]
Data set	$\left|V\right|$	$\left|E\right|$	$\left|V\right|$	$\left|E\right|$
Image Seg.	$27.7\%$	$27.4\%$	$63.7\%$	$62.7\%$
Knott-3D-150	$9.7\%$	$9.6\%$	$75.2\%$	$88.3\%$
Knott-3D-300	$54.8\%$	$61.6\%$	$76.7\%$	$91.6\%$
Knott-3D-450	$66.9\%$	$77.6\%$	$77.6\%$	$92.4\%$
Knott-3D-550	$67.8\%$	$79.0\%$	$77.8\%$	$92.6\%$
Mod. Clustering	$88.7\%$	$80.6\%$	$92.0\%$	$85.1\%$
CREMI-small	$33.8\%$	$31.9\%$	$76.6\%$	$75.3\%$
CREMI-large	$44.0\%$	$44.2\%$	$83.7\%$	$86.6\%$
Fruit-Fly Level 1–4	$8.7\%$	$9.6\%$	$24.6\%$	$27.9\%$
Fruit-Fly Global†	$56.3\%$	$51.8\%$	$77.9\%$	$74.5\%$

Table 3. Table 3: For each dataset, the table reports the average fraction of remaining nodes and edges after applying our method, respectively the QPBO method [ 34 ] (lower is better). Note that the latter is not applicable to original max-cut instances due to symmetries.

	Our		[34]
Data set	$\left|V\right|$	$\left|E\right|$	$\left|V\right|$	$\left|E\right|$
Ising Chain	$0.0\%$	$0.0\%$	n/a	n/a
2D Torus	$23.6\%$	$27.9\%$	n/a	n/a
3D Torus	$94.8\%$	$98.1\%$	n/a	n/a
Deconvolution	$61.0\%$	$56.5\%$	$61.0\%$	$56.5\%$
Super Resolution	$0.0\%$	$0.0\%$	$0.2\%$	$0.1\%$
Texture Restoration	$98.4\%$	$98.5\%$	$58.8\%$	$57.3\%$

Equations

$$\min$$

$$M=\bigcup _{1\le i<j\le k}\delta (U_i,U_j),$$

$$(\text{mathbbmss}{1}_F{)}_e=\{\begin{array}{ll}1& \text{if }e\in F\\ 0& \text{else.}\end{array}$$

$$\mathsf{MC}:=$$

$$\mathsf{CUT}:=$$

$$\min$$

$$⟨\theta ,p(x)⟩\le ⟨\theta ,x⟩\forall x\in X$$

$$p(x{)}_i=\beta \forall x\in X,$$

$$p_{\delta (U)}(x)=x\vee \text{mathbbmss}{1}_{\delta (U)}.$$

$$p_U(x{)}_{uv}=\{\begin{array}{ll}0,& uv\in E(U)\\ 0,& \exists uv\text{-path }P\text{ such that }\\ & \forall e\in E(P):\\ & x_e=0\text{ or }e\in E(U)\\ x_{uv},& \text{otherwise}.\end{array}$$

$$p_{\delta (U)}^{△}(x)=x△\text{mathbbmss}{1}_{\delta (U)}.$$

$${\theta }_f\ge \sum _{e\in \delta (U)\setminus \{f\}}|{\theta }_e|,$$

$$|{\theta }_f|\ge \sum _{e\in \delta (U)\cap E^{+}}{\theta }_e,$$

$$|{\theta }_f|\ge \sum _{e\in \delta (U)\setminus \{f\}}|{\theta }_e|,$$

$$p(x)=\{\begin{array}{ll}{p}^y(x)& \text{if }{x}_{|E_H}=y\\ x& \text{else}\end{array}$$

$${\theta }_{uw}+{\theta }_{uv}\ge \sum _{e\in \delta (U)\backslash \{uw,uv\}}|{\theta }_e|$$

$${\theta }_{uw}+{\theta }_{vw}\ge \sum _{e\in \delta (W)\backslash \{uw,vw\}}|{\theta }_e|$$

$${\theta }_{uw}+{\theta }_{uv}+{\theta }_{vw}\ge \sum _{e\in \delta (\{u,v,w\})\cap E^{+}}{\theta }_e$$

$$\underset{y\in \mathsf{MC}(H)}{\min }⟨\theta ,y⟩=0$$

$$\sum _{e\in \delta (U,V_H\setminus U)}{\theta }_e\ge \sum _{e\in \delta (V_H)\cap E^{+}}{\theta }_e,$$

$${\theta }_f\ge \sum _{e\in \delta (f)\cap E^{+}}{\theta }_e.$$

$$\min \{{\theta }_{uv}+{\theta }_{uw},{\theta }_{uv}+{\theta }_{vw},{\theta }_{uw}+{\theta }_{vw}\}$$

$$\ge \sum _{e\in \delta (\{u,v,w\})\cap E^{+}}{\theta }_e.$$

$$\sum _{e\in \delta (U,V_H\setminus U)}{\theta }_e$$

$$\displaystyle\geq\min\Bigg{\{}\sum_{e\in\delta(U,V\setminus V_{H})}\lvert\theta_{e}\rvert,\sum_{e\in\delta(V_{H}\setminus U,V\setminus V_{H})}\lvert\theta_{e}\rvert\Bigg{\}},$$

$$\underset{\hat{x}}{\min }$$

$$\sum _{e\in C}{\hat{x}}_e\ge 1,\forall \text{ conflicted }C$$

$$\hat{x}\in \{0,1{\}}^E.$$

$$\underset{\lambda }{\max }$$

$$\sum _{C:e\in C}{\lambda }_C\le |{\theta }_e|$$

$$\lambda \ge 0.$$

$$\displaystyle\tilde{\theta}_{e}:=\bigg{(}\lvert\theta_{e}\rvert-\sum_{C:e\in C}\lambda_{C}\bigg{)}\operatorname{sign}\theta_{e}\qquad\forall e\in E.$$

$$0\le \sum _{e\in \delta (U)}{\tilde{\theta }}_e\le \sum _{e\in \delta (U)}{\theta }_e.$$

$$\displaystyle\min_{\begin{subarray}{c}U\subset V_{H}:\\ u\in U,v\notin U\end{subarray}}\Bigg{(}\sum_{e\in\delta(U,V_{H}\setminus U)}\tilde{\theta}_{e}$$

Full text

1 Introduction

Partitioning graphs into meaningful clusters is a fundamental problem in combinatorial optimization with numerous applications in computer vision, biomedical image analysis, machine learning, data mining and beyond. The multicut problem (a.k.a. correlation clustering) and max-cut problem are arguably among the most well-known combinatorial optimization problems for partitioning graphs. They enable graph clustering purely based on costs between pairs of nodes and are thus commonly employed to model image processing and segmentation tasks occurring in computer vision [34, 3, 22, 18, 5]. The following factors contribute to the significance of the multicut and max-cut problem: The former allows for a graph clustering formulation that determines the number of clusters as part of the optimization process. The latter is essentially equivalent to binary quadratic programming, which has a variety of applications in image processing. However, as computer vision models are typically large-scale, standard solution techniques based on solving LP-relaxations do not scale well enough and are thus inapplicable. Even more so, finding globally optimal solutions with branch-and-cut is infeasible with off-the-shelf commercial solvers. Hence, the need arises for developing specialized heuristic solvers that output high-quality solutions for real-world problems, despite the worst-case NP-hardness of the multicut and max-cut problem. Unfortunately, although heuristic solvers often achieve a good empirical performance, they usually come without any optimality guarantees. Specifically, even if large parts of the variable assignments computed by a heuristic agree with globally optimal solutions, such optimality is not recognized.

In this work we consider combinatorial techniques for the multicut and max-cut problem by which we can efficiently find persistency (a.k.a. partial optimality). Persistent variable assignments come with a certificate that proves their agreement with a globally optimal solution. The potential benefits are twofold: (i) After running a primal heuristic, we can compute certificates which show that some variables are persistent. (ii) Even before running a heuristic, we may determine in a preprocessing step persistent variable assignments. In either case, the problem size can be reduced. In the first case, a subsequent optimization with exact solvers is accelerated. In the second case, possibly also the runtime of a heuristic algorithm is reduced and the solution quality improved.

A joint treatment of the multicut and max-cut problem seems instructive, since many criteria have a similar formulation and are based on analogous arguments. For the max-cut problem we offer, to our knowledge, a novel approach for computing persistent variable assignments. For the multicut problem our empirical evidence suggests that our method offers substantial improvement over prior work on persistency. Our empirical results are most significant for very large scale problems which current heuristics can barely handle, e.g. in biomedical image segmentation [5]. By reducing problem size via persistency, our method enables high quality solutions in such cases.

The paper is organized as follows. In Section 2 we review the related work. In Section 3 we introduce the multicut and max-cut problem mathematically in a shared compact formulation. In Section 4 we recap the concept of improving mappings in the context of persistency. Further, we introduce fundamental building blocks for the construction of improving mappings for the multicut and max-cut problem. In Section 5 and 6 we present our combinatorial persistency criteria and devise algorithms to check them. Finally, in Section 7 we evaluate our methods in numerical experiments on instances from the literature and compare to related work. The more technical proofs for our results are provided in the appendix. In the appendix we also present technical improvements of our persistency criteria that were omitted from the main paper for the sake of clarity.

2 Related work

Persistency for Markov Random Fields (MRF) and, as a special case, for the binary quadratic optimization problem (a.k.a. Quadratic Pseudo-Boolean Optimization (QPBO)), has been well studied. It was observed in [31] that a natural LP-relaxation of the stable set problem has the persistency property: All integral variables of LP-solutions coincide with a globally optimal one. This result has been transferred to QPBO [16, 6, 7] and extended in [45] to find relational persistency, i.e. showing that some pairs of variables must have the same/different values. For higher order binary unrestricted optimization problems, the concept of roof duality can be extended to obtain further persistency results [34, 19, 26]. Going beyond the basic LP-relaxation for QPBO, persistency certificates involving tighter LP-relaxations for higher order polynomial 0/1-programs that do not possess the persistency property (i.e. integral variables need not be persistent) have been studied in [1].

For general MRFs, criteria that can be elementarily checked include Dead End Elimination (DEE) [12]. More powerful techniques generalizing DEE that still can be used for fast preprocessing can be found in [44]. The MQPBO method [24] consists of transforming multilabel MRFs to the QPBO problem and persistency results from QPBO can subsequently be used to obtain persistency for the original multilabel MRF. Persistency criteria for the multilabel Potts problem that can be efficiently checked with max-flow computations have been developed in [27, 28] and refined in [14]. More powerful criteria based on LP-relaxations have been proposed for the multilabel Potts problem in [41] and in [38, 42, 40] for general discrete MRFs. An in-depth exposition of the concept of improving mappings that is used implicitly or explicitly for all of the above MRF criteria can be found in [37]. A comprehensive theoretical discussion and comparison of the above persistency techniques can be found in [39].

There has been, to our knowledge, less work on persistency for the multicut and max-cut problem. For multicut, the works [2, 29] proposed simple persistency criteria that allow to fix some edge assignments. We are not aware of any persistency results for max-cut. Also it is not easily possible to transfer persistency results from QPBO to max-cut, even though there exist straightforward transformations between these two problems. The underlying reason is that the transformation from max-cut to QPBO introduces symmetries which current persistency criteria cannot handle. More specifically, known persistency criteria rely on an improving mapping, but in symmetric instances it is always possible to map a labeling to an equivalent one with the same cost by exploiting symmetries. Consequently, fixed-points of improving mappings, which amount to persistent variables, cannot be found. For the closely related (yet polynomial-time solvable) min-cut problem, a family of persistency criteria were proposed in [32, 17]. They directly translate to the max-cut problem and we derive them as special cases in our study below.

The more involved constraints describing the multicut and max-cut problem make it difficult to directly transfer some of the powerful persistency techniques that are available for MRFs. In our work we show how the framework of improving mappings developed in [37] can be used to derive persistency criteria for combinatorial problems with more complicated constraint structures, such as the multicut and max-cut problem, once a class of mappings that act on feasible solutions is identified. Specifically, we show that the known multicut persistency criteria from [29] and the persistency criteria from [17] (transferred to the max-cut problem) can be derived in our theoretical framework. Moreover, we define more powerful criteria that can find significantly more persistent variables, as shown in the experimental Section 7, yet can be evaluated efficiently. We believe that our approach of composing improving mappings from elementary mappings is instructive in the search for more persistency criteria.

3 Multicut and max-cut

Let

[TABLE]

with $X\subseteq\{0,1\}^{m}$ be a linear combinatorial optimization problem. In this paper, we study specific instances of (P) known as the multicut and the max-cut problem, which are introduced mathematically in this section. To this end, let $G=(V,E,\theta)$ be a weighted graph, where $\theta\in\mathbb{R}^{E}$. We distinguish non-negative and negative edges via $E=E^{+}\cup E^{-}$ with $E^{+}=\{e\in E\mid\theta_{e}\geq 0\}$ and $E^{-}=\{e\in E\mid\theta_{e}<0\}$. For any two disjoint subsets of vertices $U,W\subseteq V$ let $\delta(U,W)=\{uw\in E\mid u\in U,w\in W\}$ denote the set of edges between $U$ and $W$. Further, we write $\delta(U):=\delta(U,V\setminus U)$. For any subgraph $H=(V_{H},E_{H})$ of $G$ we may identify $H$ with $E_{H}$ and write $e\in H$ instead of $e\in E_{H}$.

Definition 1** (Multicuts and Cuts).**

Let $(U_{1},\ldots,U_{k})$ be a partition of $V$, i.e. $U_{1}\cup\ldots\cup U_{k}=V$ and $U_{i}\cap U_{j}=\varnothing$ for $i\neq j$. The set of edges $M$ between any pair of components of the partition, defined by

[TABLE]

is called a multicut of $G$. If $k=2$, then $M=\delta(U_{1})=\delta(U_{2})$ is called a cut of $G$. For any set of edges $F\subseteq E$ define the incidence vector $\mathbbmss{1}_{F}\in\{0,1\}^{E}$ of $F$ via

[TABLE]

We write

[TABLE]

for the set of incidence vectors of multicuts, respectively cuts of $G$.

**Multicut.  ** The multicut problem is to find a multicut of minimum weight w.r.t. $\theta$ and can be written as an instance of (P) as follows:

[TABLE]

**Max-Cut.  ** The max-cut problem is to find a cut $\delta(U)$, $U\subseteq V$, of maximum weight (or equivalently of minimum weight for $-\theta$). After setting $\theta\leftarrow-\theta$ it can be written as an instance of (P) as follows:

[TABLE]

Note that we use $\min$ instead of $\max$ to conform to (P).

4 Improving mappings

In this section, we introduce improving mappings as a concept to derive partial optimality results and define elementary building blocks to construct improving mappings for the multicut and max-cut problem.

Definition 2** ([38]).**

A mapping $p\colon X\rightarrow X$ with the property

[TABLE]

is called improving mapping.

An improving mapping $p$ that maps some variable $i\in[m]$ to a fixed value $\beta$ provides persistency (a.k.a. partial optimality): For each feasible element $x\in X$, a better one is obtained by applying $p$ to $x$ and thus fixing $x_{i}=\beta$.

Lemma 1** (Persistency).**

Let $p\colon X\to X$ be an improving mapping and $\beta\in\{0,1\}$. If

[TABLE]

then $x^{*}_{i}=\beta$ in some optimal solution $x^{*}$ of (P).

Proof.

Let $x$ be an optimal solution of (P). Then $x^{*}=p(x)$ is also optimal and $x^{*}_{i}=\beta$. ∎

There are two trivial improving mappings: (i) The identity mapping $\operatorname{id}\colon x\mapsto x$. It does not provide any persistency at all. (ii) The mapping $p^{*}\colon x\mapsto x^{*}$ that maps any $x$ to a fixed optimal solution $x^{*}\in\operatorname*{argmin}_{x\in X}\langle\theta,x\rangle$. This mapping obviously provides the maximal persistency, but for NP-hard problems it is generally intractable to compute $x^{*}$.

We are hence interested in a middle ground: We want to find improving mappings that fix as many variables as possible (unlike $\operatorname{id}$) but that are computable in polynomial time (unlike $p^{*}$). This allows us to simplify the original problem (P) by fixing the persistent variables. For the multicut problem we can contract those edges that can be persistently set to [math], which allows to shrink the underlying graph. For the max-cut problem, however, any value for persistent variables can be exploited for contractions, as we show below.

4.1 Elementary mappings

In order to construct improving mappings for the multicut and max-cut problem, we employ the elementary mappings defined in this section.

Definition 3** (Multicut mappings).**

Let $U\subseteq V$ be a set of nodes that induce a connected component of $G$.

• (i)

The elementary cut mapping $p_{\delta(U)}$ is defined as

[TABLE]

In other words, this means that $p_{\delta(U)}(x)_{e}=1$ for all edges $e\in\delta(U)$ and $p_{\delta(U)}(x)_{e}=x_{e}$ otherwise.

• (ii)

The elementary join mapping $p_{U}$ is defined as

[TABLE]

Intuitively, the elementary cut mapping $p_{\delta(U)}$ adds the cut $\delta(U)$ to the multicut defined by $x$. The elementary join mapping $p_{U}$ merges all components that intersect with $U$, cf. Figure 1. To show well-definedness of the elementary cut and join mapping rigorously, we need the following characterization of multicuts.

Fact 1** ([9]).**

A set $M\subseteq E$ is a multicut iff for every cycle $C$ of $G$ it holds that $\lvert M\cap C\rvert\neq 1$.

Lemma 2** (Well-definedness).**

The mappings $p_{\delta(U)}$ and $p_{U}$ are well-defined, i.e.

• (i)

$p_{\delta(U)}\colon\mathsf{MC}\rightarrow\mathsf{MC}$* for any connected $U\subseteq V$*

• (ii)

$p_{U}\colon\mathsf{MC}\rightarrow\mathsf{MC}$* for any connected $U\subseteq V$.*

The elementary mapping for the max-cut problem exploits the well-known property of cuts that they are closed under taking symmetric differences (of edges).

Fact 2** ([36]).**

Let $x,y\in\mathsf{CUT}$. Then $x\triangle y\in\mathsf{CUT}$.

In particular, since $x\mapsto x\triangle y$ is an involution (i.e. its own inverse) for any cut $y\in\mathsf{CUT}$, it holds that $\mathsf{CUT}\triangle y:=\{x\triangle y\mid x\in\mathsf{CUT}\}=\mathsf{CUT}$. Given an instance of max-cut defined by $G=(V,E,\theta)$ and a cut $y\in\mathsf{CUT}$, this transformation of the feasible set corresponds to switching the signs of $\theta_{e}$ for all $e\in E$ with $y_{e}=1$ and adding the constant $\sum_{e\in E}\theta_{e}y_{e}$ to the objective value. If $y$ is optimal for the original instance, then $y\triangle y=0$ is optimal for the transformed instance. Hence, whenever we want to compute persistency for $x_{f}=1$, we can transform the instance to an equivalent one by applying the described switching for any cut that contains $f$ and then checking whether $x_{f}=0$ holds persistently.

Definition 4** (Symmetric Difference Mapping).**

Let $U\subseteq V$. The elementary symmetric difference mapping $p^{\triangle}_{\delta(U)}$ w.r.t. $\delta(U)$ is defined as

[TABLE]

In other words, this means that $p^{\triangle}_{\delta(U)}(x)_{e}=1-x_{e}$ for all edges $e\in\delta(U)$ and $p^{\triangle}_{\delta(U)}(x)_{e}=x_{e}$ otherwise. The symmetric difference mapping is well-defined because of Fact 2. See Figure 2 for an illustration of $p^{\triangle}_{\delta(U)}$.

5 Persistency criteria

In this section, we propose subgraph-based criteria for finding improving mappings. We provide criteria for small connected subgraphs such as edges or triangles as well as criteria for general connected subgraphs. In Section 6, we present efficient heuristic algorithms to check the subgraph criteria proposed in this section.

First consider the instructive special case of a single edge subgraph. The following criterion has been evaluated by [29] for the multicut problem.

Theorem 1** (Edge Criterion).**

Let $f\in E$ be an edge and $U\subseteq V$ be connected with $f\in\delta(U)$. Further, let $\beta=(1-\operatorname{sign}\theta_{f})/2$. If

[TABLE]

then $x^{*}_{f}=\beta$ in some optimal solution $x^{*}$ of (P).

The proof of Theorem 1 relies on Lemma 1 by applying the mapping $p^{\triangle}_{\delta(U)}$, respectively $p_{f}\circ p_{\delta(U)}$, to improve any solution $x$ with $x_{f}\neq\beta$. Simple candidates for $U$ are $\{u\}$ and $\{v\}$ where $f=uv$. Checking these for every edge $f\in E$ can be done in linear time. All $u$-$v$-cuts can be checked at once by minimizing the right-hand sides of (2) – (4) via max-flow techniques on the weighted graph $G^{\lvert\cdot\rvert}=(V,E,\lvert\theta\rvert)$, respectively $G^{+}=(V,E^{+},\theta)$ for (3). Note that the condition in (3) is less restrictive than (2). Computing a Gomory-Hu tree [13] of $G^{\lvert\cdot\rvert}$ or $G^{+}$ reduces the total computational effort of checking the criterion for all edges $f\in E$ to $\lvert V\rvert-1$ max-flow problems.

5.1 General subgraph criteria

We give a technical lemma that allows to generalize the persistency criterion stated in Theorem 1.

Lemma 3**.**

Let $f\in E$ and $\beta\in\{0,1\}$. Further, let $H=(V_{H},E_{H})$ be a connected subgraph of $G$ such that $e\in E_{H}$. If for every $y\in\mathsf{CUT}(H)$ with $y_{f}=1-\beta$, there exists a mapping $p^{y}\colon X\to X$ such that for all $x\in X$ with $x_{|E_{H}}=y$ we have

1. (i)

$\langle\theta,p^{y}(x)\rangle\leq\langle\theta,x\rangle$** 2. (ii)

$p^{y}(x)_{f}=\beta$,

then $x^{*}_{f}=\beta$ in some optimal solution $x^{*}$.

Proof.

Condition (i) implies that the mapping $p\colon X\rightarrow X$ defined by

[TABLE]

is improving. Condition (ii) implies $p(x)_{e}=\beta$ for all $x$. ∎

Consider the special case when $H$ is a triangle subgraph.

Corollary 1** (Triangle Criterion).**

Let $\{uw,uv,vw\}\subset E$ be a triangle. Let $U\subset V$ be such that $uv,uw\in\delta(U)$, and $W\subset V$ be such that $uw,vw\in\delta(W)$.

1. (i)

If

[TABLE]

holds, then $x^{*}_{uw}=0$ for some optimal solution of (**PCUT***).* 2. (ii)

If additionally

[TABLE]

holds, then $x^{*}_{uw}=0$ for some optimal solution of (**PMC***).*

A straightforward choice for the cuts in Corollary 1 are $\delta(\{u\}),\delta(\{w\}),\delta(\{v,w\})$ and $\delta(\{u,v\})$, as depicted in Figure LABEL:fig:subgraph-criteria a). It is possible to find better cuts w.r.t. costs $\lvert\theta\rvert$, but we are not aware of any more efficient technique than to explicitly compute them via max-flow for every triangle (unlike computing a Gomory-Hu tree to evaluate the single edge criterion for all edges).

We further apply Lemma 3 to state general subgraph criteria for the multicut and max-cut problem. See Figure LABEL:fig:subgraph-criteria b) for a schematic illustration.

Theorem 2** (Multicut Subgraph Criterion).**

Let $H=(V_{H},E_{H})$ be a connected subgraph of $G$ and suppose $uv\in E_{H}$. If

[TABLE]

and for all $U\subset V_{H}$ with $u\in U$ and $v\notin U$ it holds that

[TABLE]

then $x^{*}_{uv}=0$ in some optimal solution $x^{*}$ of (PMC).

In the proof of Theorem 2 we use the mapping $p_{V_{H}}\circ p_{\delta(V_{H})}$ to improve solutions $x\in\mathsf{MC}$ with $x_{f}\neq 0$. Note that the multicut subgraph criterion stated in Theorem 2 is different from the edge and triangle criteria when evaluated on these special subgraphs. If $H=(f,f)$ for some edge $f\in E$, then condition (9) translates to

[TABLE]

If $H$ is a triangle, i.e. $H=(\{u,v,w\},\{uv,uw,vw\})$ for some vertices $u,v,w\in V$, then condition (9) translates to

[TABLE]

Theorem 3** (Max-Cut Subgraph Criterion).**

Let $H=(V_{H},E_{H})$ be a connected subgraph of $G$ and suppose $uv\in E_{H}$. If for all $U\subset V_{H}$ with $u\in U$ and $v\notin U$ it holds that

[TABLE]

then $x^{*}_{uv}=0$ in some optimal solution $x^{*}$ of (PCUT).

In the proof of Theorem 3 we either use the mapping $p^{\triangle}_{\delta(U)}$ or $p^{\triangle}_{\delta(V_{H}\setminus U)}$ to improve solutions $x\in\mathsf{CUT}$ with $x_{f}\neq 0$. Note that if $H$ is a single edge or a triangle, the subgraph criterion stated in Theorem 3 specializes to the edge criterion, respectively triangle criterion, where only the cuts $\delta(\{u\})$, $\delta(\{v\})$, respectively $\delta(\{u\}),\delta(\{w\}),\delta(\{v,w\})$ and $\delta(\{u,v\})$ are considered.

6 Algorithms

In this section we devise algorithms that verify, for a given instance $G=(V,E,\theta)$ of the multicut or max-cut problem, the persistency criteria presented in Section 5.

The edge and triangle criteria can be checked explicitly for all edges, respectively triangles of $G$. Note that listing all triangles of a graph can be done efficiently [35].

Therefore, we focus here on developing efficient algorithms that find subgraphs $H$ which qualify for the criteria from Theorem 2 and 3. Specifically, we propose routines that (i) check for a given connected subgraph $H$ whether some persistency criteria apply and (ii) find good candidates for $H$.

6.1 Subgraph evaluation

Let $H=(V_{H},E_{H})$ be a subgraph of $G$ that we want to check for persistency condition (9), respectively (10). Now, for a given edge $uv\in E_{H}$, we can determine if (9) holds true for all $U\subset V_{H}$ with $u\in U$ and $v\notin U$ by minimizing the left-hand side w.r.t. $U$. In contrast, for (10), we also need to simultaneously maximize the right-hand side, since it depends on $U$ as well. Obviously, minimizing the left-hand side (of either (9) or (10)) means finding a minimum $u$-$v$-cut w.r.t. $\theta$. Further, since the right-hand sides are non-negative, the minimum $u$-$v$-cut must have non-negative weight. However, in general the weights $\theta$ on $H$ may be negative, which renders both optimization problems hard in general.

For this reason, we simplify the problem by restriction to suitable subgraphs $H$ that satisfy Assumption 1 below. In order to state Assumption 1 rigorously, we need to briefly recap the following integer linear programming (ILP) formulation of the multicut problem.

The multicut problem can be stated equivalently to (PMC) as finding the minimum weight edge set w.r.t. $\lvert\theta\rvert$ that covers every cycle with exactly one negative edge, the so-called erroneous or conflicted cycles [11, 29]. The corresponding ILP formulation reads

[TABLE]

The associated packing dual is the linear program

[TABLE]

For any dual feasible $\lambda\geq 0$, the associated reduced costs for the primal problem (11) are given by

[TABLE]

Assumption 1**.**

Let $H=(V_{H},E_{H},\theta)$ be a weighted graph such that

1. i)

The graph $H$ has a trivial multicut solution, i.e. $\min_{y\in\mathsf{MC}(H)}\langle\theta,y\rangle=0$. 2. ii)

An optimal packing dual solution $\lambda^{*}$ for the multicut problem on $H$ is available.

Note that Assumption 1 i also implies a trivial max-cut solution, since $\mathsf{CUT}\subseteq\mathsf{MC}$. Assumption 1 has the following expedient consequence.

Lemma 4**.**

Let $H=(V_{H},E_{H},\theta)$ be a weighted graph that satisfies Assumption 1. Then $\tilde{\theta}_{e}\geq 0$ for all $e\in E_{H}$ and for any cut $\delta(U)$ of $H$ it holds that

[TABLE]

Our method exploits Assumption 1 and Lemma 4 as follows. First, we compute a solution to the packing dual (13) by the fast heuristic Iterative Cycle Packing (ICP) algorithm from [29]. Then, if the computed dual solution shows that the multicut solution on $H$ is trivial, we can compute lower bounds to the left-hand side of (9) and (10) by applying max-flow techniques on $H$ with capacities $\tilde{\theta}$.

In the case of the multicut problem, the right-hand side of (9) is constant w.r.t. $U$ so it suffices to compute a Gomory-Hu tree on $H$. In the case of the max-cut problem, however, this is not sufficient, since the right-hand side of (10) also depends on $U$. Here, after replacing $\theta_{e}$ by $\tilde{\theta}_{e}$ for all $e\in E_{H}$, we need to solve the following $\min\max$ problem

[TABLE]

As solving this problem exactly appears to be difficult, we propose to solve a relaxation that is obtained by replacing the inner $\max$ term with

[TABLE]

and then swapping the order of $\min$ and $\max$. This yields

[TABLE]

The right-hand side is the maximization of a concave, non-smooth function on the unit interval, which can be solved efficiently with the bisection method. In every iteration, the inner minimization problem needs to be solved for a fixed $\alpha\in[0,1]$, which can be formulated again as a max-flow problem.

For solving the max-flow problems that occur in our method, we use Boykov-Kolmogorov’s algorithm with reused search trees [8, 25]. For computing Gomory-Hu trees, we use a parallelized implementation of Gusfield’s algorithm [15, 10].

6.2 Finding candidate subgraphs

To efficiently find good candidate subgraphs, we employ the following strategy. First, we compute a primal feasible solution $\bar{x}\in X$ by a fast heuristic method such as greedy edge contraction algorithms [20, 23]. If the heuristic solution $\bar{x}$ is reasonably good, then many components defined by $\bar{x}$ should be close to optimal. Thus, in the case of the multicut problem, the components may already serve as candidate subgraphs. In the case of the max-cut problem, we use $\bar{x}$ to transform the instance by the switching operation described in Section 4.

Then, we compute a heuristic packing dual solution $\bar{\lambda}$ by ICP for the entire graph $G=(V,E,\theta)$. The candidate subgraphs are determined as the connected components of the positive residual graph $(V,\{e\in E\mid\tilde{\theta}_{e}>0\})$, where $\tilde{\theta}$ is defined similarly as before in (14). The intuition behind this strategy is that, by construction, the subgraphs’ inner edges have relatively higher weight than their outgoing edges. This facilitates the application of the conditions (9) and (10).

**Reduced cost fixing.  ** Further, whenever both a primal solution and dual solution are available, we use the following technique known as reduced cost fixing [4] to determine additional persistent variables. Let $\gamma:=\langle\theta,\bar{x}\rangle-\langle\mathbbmss{1},\bar{\lambda}\rangle-\sum_{e\in E^{-}}\theta_{e}$ denote the duality gap of the primal-dual solution pair and suppose $\gamma<\tilde{\theta}_{f}$ for some $f\in E$. Then, it follows that $x_{f}=1$ cannot be optimal and thus we can fix $x_{f}=0$.

7 Experiments

In order to study the effectiveness of our methods, we evaluate them on a collection of more than 200 instances from the literature. The size of the instances ranges from a few hundred to hundreds of millions of variables (edges). As a performance measure, we use the average relative size reduction of test instances that is obtained by applying our algorithms.

**Instances.  ** For the multicut problem we use segmentation and clustering instances from the OpenGM benchmark [21] as well as biomedical segmentation instances provided by the authors of [5] and [33]. The dataset Image Segmentation contains planar graphs that are constructed from superpixel adjacencies of photographs. The Knott-3D data sets contains non-planar graph arising from volume images acquired by electron microscopy. The set Modularity Clustering contains complete graphs constructed from clustering problems on small social networks. The CREMI data sets contain supervoxel adjacency graphs obtained from volume image scans of neural tissue. The Fruit-Fly instances were generated from volume image scans of fruit fly brain matter. The global problem is the largest instance in this study with roughly 650 million variables. It represents the current limit of what can be tackled by state-of-the-art local search algorithms. The instances Level 1–4 are progressively simplified versions of the global problem obtained via block-wise domain decomposition [33].

For the max-cut problem we use two different types of instances. (i) The datasets Ising Chain, 2D Torus and 3D Torus contain instances that stem from applications in statistical physics [30]. The instances in Ising Chain assume a linear order on the nodes. For any pair of nodes there is an edge with an associated weight. The absolute values of the weights decrease exponentially with the distance of the nodes in the linear order. The instances in 2D Torus and 3D Torus are defined on toroidal grid graphs in two, resp. three dimensions with Gaussian distributed weights. (ii) The datasets Deconvolution, Super Resolution and Texture Restoration contain QPBO instances originating from image processing applications [34, 43] that are converted to our formulation of the max-cut problem. The transformation introduces an additional node that is connected with all other nodes. A cut (uncut) edge to the additional node signifies label 0 (resp. 1). The instance size statistics for all data sets are summarized in Table 1.

**Results.  ** In Table 2, we report for the multicut instances the average graph sizes after shrinking the instances with our algorithms from Section 6. In Figure 4, the contributions of the individual persistency criteria are separated and compared to prior work. It can be seen from Table 2 and Figure 4 that our criteria enable finding substantially more persistent variables than the prior work [2, 29]. In relation to the graph sizes after shrinking with the baseline [29], our method achieves an additional size reduction of about 30–60% for the large CREMI and Fruit-Fly instances. This shows that our algorithms find persistent variable assignments that are harder to detect than with the criteria from prior work.

In Table 3 we report for the max-cut instances the average graph size reduction on each dataset. For the QPBO instances we compare to the QPBO method [34]. For the original max-cut instances we are unaware of any baseline method and the QPBO method is not applicable. In Figure 5 we compare the contribution of the different subgraph criteria. It can be seen that our method solves all Ising Chain instances to optimality, which is facilitated by their particular distribution of the weights. On 2D Torus we achieve substantial size reductions and on the denser 3D Torus instances we find few persistencies. Our results on the QPBO instances are on a par with [34] for Deconvolution and Super Resolution while our method is less effective for Texture Restoration.

8 Conclusion

We have presented combinatorial persistency criteria for the multicut and max-cut problem. Moreover, we have devised efficient algorithms to check our criteria. For multicut our method achieves a substantial improvement over prior work when evaluated on common benchmarks as well as practical instances. For max-cut we are, to the best of our knowledge, the first to propose an algorithm that computes persistent variable assignments for the general problem. For the special case of QPBO problems, our method matches the performance of prior work on some instances. Our results demonstrate the feasibility of computing persistent variable assignments for NP-hard graph cut problems in practice. Besides acquiring partial optimality guarantees, our approach is a helpful tool for shrinking problem sizes and thus essential toward identifying globally optimal solutions.

Appendix A Proofs

Proof of Lemma 2.

Proof.

(i) Let $x\in\mathsf{MC}$ and assume that $z=p_{\delta(U)}(x)\notin\mathsf{MC}$. Then there exists a cycle $C$ with exactly one cut edge in $z$, i.e. $z_{f}=1$ for some $f\in C$ and $z_{e}=0$ for all $e\in C\setminus f$. It holds that $x_{f}=1$ and thus $C$ crosses $\delta(U)$ exactly once, which is impossible.

(ii) Let $x\in\mathsf{MC}$ and assume that $z=p_{U}(x)\notin\mathsf{MC}$. Then there is a cycle $C$ with $z_{f}=1$ for some $f\in C$ and $z_{e}=0$ for all $e\in C\setminus f$. Since $z\leq x$ there exists an edge $uv=g\in C$, $g\neq f$ with $x_{g}=1$ and $z_{g}=0$. Then, according to (1), there exists a $uv$-path $P$ such that $x_{e}=0$ for all $e\in P\setminus E(U)$. Replace the cycle $C$ by $C\triangle(P\cup\{g\})$. Repeating this argument for all such edges $g\in C$ yields a path $P^{\prime}$ connecting the endpoints of $f$ such that $x_{e}=0$ for all $e\in P^{\prime}\setminus E(U)$, which is a contradiction to $z_{f}=1$. ∎

Proof of Theorem 1.

Proof.

First, we show that the mapping

[TABLE]

is improving for the max-cut problem. Let $x\in\mathsf{CUT}$, $z=p(x)$ and suppose $x_{f}\neq\beta$. It holds that

[TABLE]

Similarly, for $\beta=0$, we show that the mapping

[TABLE]

is improving for the multicut problem. Let $x\in\mathsf{MC}$, $z=p(x)$ and suppose $x_{f}\neq\beta$. It holds that

[TABLE]

Finally, for $\beta=1$, we show that the mapping

[TABLE]

is improving for multicut. Let $x\in\mathsf{MC}$, $z=p(x)$ and suppose $x_{f}\neq\beta$. It holds that

[TABLE]

This concludes the proof. ∎

Proof of Corollary 1.

Proof.

We use Lemma 3:

1. (i)

In the case $x_{uw}=1$, $x_{uv}=1$, $x_{vw}=0$ apply $p^{\triangle}_{\delta(U)}$. In the case $x_{uw}=1$, $x_{uv}=0$, $x_{vw}=1$ apply $p^{\triangle}_{\delta(W)}$. These mappings are improving due to (5) and (6). 2. (ii)

In the case $x_{uw}=1$, $x_{uv}=1$, $x_{vw}=0$ apply $p_{\{u,w\}}\circ p_{\delta(U)}$. In the case $x_{uw}=1$, $x_{uv}=0$, $x_{vw}=1$ apply $p_{\{u,w\}}\circ p_{\delta(W)}$. These mappings are improving analogously to (i). In the additional case $x_{uw}=1$, $x_{uv}=1$, $x_{vw}=1$ apply the mapping $p=p_{\{u,v,w\}}\circ p_{\delta(\{u,v,w\})}$. It is improving, since

[TABLE]

Proof of Theorem 2.

Proof.

We use Lemma 3. Let $y\in\mathsf{MC}(H)$ with $y_{uv}=1$ and suppose $x\in\mathsf{MC}$ with $x_{|E_{H}}=y$. Then there is a multicut $M$ of $H$ such that $y=\mathbbmss{1}_{M}$. Due to (8), every (multi-)cut of $H$ has nonnegative weight. Therefore, there exists some $U\subset V_{H}$ with $u\in U$ and $v\notin U$ such that $\delta(U,V_{H}\setminus U)\subseteq M$ and

[TABLE]

Let $p^{y}(x)=(p_{V_{H}}\circ p_{\delta(V_{H})})(x)$, then it follows from (9) and (17) that

[TABLE]

Proof of Theorem 3.

Proof.

We use Lemma 3. Suppose $y\in\mathsf{CUT}(H)$ with $y_{uv}=1$. Let $U\subset V_{H}$ be such that $y$ is the incidence vector of $\delta(U,V_{H}\setminus U)$ in $H$ and suppose $x\in\mathsf{CUT}$ with $x_{|E_{H}}=y$. We may assume that

[TABLE]

otherwise redefine $U:=V_{H}\setminus U$. Now, let $z=p^{y}(x)=p^{\triangle}_{\delta(U)}(x)$, then it follows that

[TABLE]

Proof of Lemma 4.

Proof.

As $H$ satisfies Assumption 1, the dual problem (13) evaluates to zero. Thus, since any conflicted cycle contains exactly one edge $e\in E_{H}$ with $\theta_{e}<0$, we must have $\sum_{C:e\in C}\lambda^{*}_{C}=\lvert\theta_{e}\rvert$, which implies $\tilde{\theta}_{e}=0$. Furthermore, for any cut $\delta(U)$ of $H$ it holds that

[TABLE]

The last inequality holds true, because every cycle with precisely one negative edge $e$, where $e\in\delta(U)\cap E^{-}$, also contains some positive edge $f\in\delta(U)\cap E^{+}$, as it crosses $\delta(U)$ at least twice. This concludes the proof. ∎

Appendix B Persistency criteria

In this section we describe a technical improvement of the multicut subgraph criterion presented in Theorem 2. Here, improvement means relaxing the inequality (9) such that it applies more often (without compromising the persistency result).

To this end, we need to introduce some more notation. For any set of vertices $U\subseteq V$, let

[TABLE]

denote the boundary of $U$ in $V$. The boundary of $U$ consists of those vertices in $V$ that have a neighbor in $U$ but are not in $U$ themselves. For any set $U\subseteq V$, its closure ${}\mkern 2.0mu\overline{\mkern-2.0muU\mkern-1.0mu}$ is defined as the union of $U$ with its boundary, i.e.

[TABLE]

Further, for any connected subgraph $H=(V_{H},E_{H})$, we define its positive closure as the subgraph

[TABLE]

which additionally includes all positive edges between $V_{H}$ and its boundary.

Below we state a more refined version of Theorem 2. The difference in Theorem 4 is that the inner cut is w.r.t. the subgraph ${}\mkern 2.0mu\overline{\mkern-2.0muH\mkern-1.0mu}$ instead of $H$.

Theorem 4** (Multicut Subgraph Criterion).**

Let $H=(V_{H},E_{H})$ be a connected subgraph of $G$ and suppose $uv\in E_{H}$. If

[TABLE]

and for all $U\subset{}\mkern 2.0mu\overline{\mkern-2.0muV_{H}\mkern-1.0mu}$ with $u\in U$ and $v\notin U$ it holds that

[TABLE]

then $x^{*}_{uv}=0$ in some optimal solution $x^{*}$ of (PMC).

Proof.

The proof is largely analogous to the proof of Theorem 2. Suppose $x\in\mathsf{MC}$ with $x_{|E_{H}}=y\in\mathsf{MC}(H)$ and $y_{uv}=1$. Apparently, there exists a multicut $M$ of ${}\mkern 2.0mu\overline{\mkern-2.0muH\mkern-1.0mu}$ which extends $y$ such that $x_{|E_{{}\mkern 2.0mu\overline{\mkern-2.0muH\mkern-1.0mu}}}=\mathbbmss{1}_{M}$. Similarly to before, there exists some $U\subset{}\mkern 2.0mu\overline{\mkern-2.0muV_{H}\mkern-1.0mu}$ with $u\in U$ and $v\notin U$ such that $\delta(U,{}\mkern 2.0mu\overline{\mkern-2.0muV_{H}\mkern-1.0mu}\setminus U)\cap E_{{}\mkern 2.0mu\overline{\mkern-2.0muH\mkern-1.0mu}}\subseteq M$ and

[TABLE]

Eventually, using (19) and (20), we show that the mapping $p^{y}=(p_{V_{H}}\circ p_{\delta(V_{H})})$ still improves $x$, as follows:

[TABLE]

The inequality (19) is less restrictive than (9) in Theorem 2, because the left-hand side is potentially larger. Indeed, if two neighboring nodes $u,v\in V_{H}$ are connected by positive edges to some vertex $w\in\partial V_{H}$ in the boundary (i.e. they form a positive triangle), then the extension of any cut that separates $u$ from $v$ has to cut another edge of the triangle. Thus, the weight of this edge can be subtracted from the right-hand side of the inequality (9) or, equivalently, added to the left-hand side, which is what (19) achieves.

For the special case of a single edge subgraph $H=(\{u,v\},\{uv\})$ the refined condition is explicitly stated as

[TABLE]