Multi-way sparsest cut problem on trees with a control on the number of parts and outliers

TL;DR

This paper introduces a polynomial-time dynamic programming algorithm for a generalized multi-way sparsest cut problem on trees, balancing cluster edge expansion and outlier control.

Contribution

It extends the sparsest cut problem to multiple clusters with outlier constraints on trees, providing an efficient solution despite NP-hardness in general.

Findings

Polynomial-time algorithm for weighted trees with connected subgraph restriction

Algorithm runs in O(k^2 n^3) worst case complexity

Linear time complexity when clusters and outliers are bounded by constants

Abstract

Given a graph, the sparsest cut problem asks for a subset of vertices whose edge expansion (the normalized cut given by the subset) is minimized. In this paper, we study a generalization of this problem seeking for $ k $ disjoint subsets of vertices (clusters) whose all edge expansions are small and furthermore, the number of vertices remained in the exterior of the subsets (outliers) is also small. We prove that although this problem is $ NP-$hard for trees, it can be solved in polynomial time for all weighted trees, provided that we restrict the search space to subsets which induce connected subgraphs. The proposed algorithm is based on dynamic programming and runs in the worst case in $ O(k^2 n^3) $, when $ n $ is the number of vertices and $ k $ is the number of clusters. It also runs in linear time when the number of clusters and the number of outliers is bounded by a constant.

Equations

$$\varphi (G):=\underset{S⊊V(G),S\ne \varnothing }{\min }\max \left\{\frac{|\partial S|}{|S|},\frac{|\partial S|}{|\overline{S}|}\right\},$$

$${\varphi }_G(S):=\frac{c(\partial S)}{\omega (S)},$$

$$\omega (S):=\sum _{u\in S}\omega (u),c(\partial S):=\sum _{e\in E(S,\overline{S})}c(e).$$

$${\iota }_k(G)$$

$${\tilde{\iota }}_k(G)$$

$${\varphi }_T(A_i):=\frac{c(\partial A_i)}{\omega (A_i)},{\varphi }_T(\mathcal{A}):=\underset{1\le i\le k}{\max }{\varphi }_T(A_i)\mathrm{and}{\iota }_k^C(T):=\underset{\mathcal{A}\in {\mathcal{C}}_k(T)}{\min }{\varphi }_T(\mathcal{A}).$$

$${\epsilon }_{\xi }(e):=\xi \omega (T_e)+c(e).$$

$${\gamma }_{\xi }(\mathcal{A}):=\sum _{e\in F_{\mathcal{A}}}{\epsilon }_{\xi }(e).$$

$${\Gamma }_{\xi }(u,k,l):=\underset{\mathcal{A}\in {\mathcal{C}}_{\xi }(u,k,l)}{\min }{\gamma }_{\xi }(\mathcal{A}).$$

$${\mu }_{\xi }(u,0,l)=\{\begin{array}{ll}1& \text{ if }|V(T_u)|\le l,\\ 0& \text{ o.w.}\end{array}$$

$$U(1,k,l):={\mu }_{\xi }(u_1,k,l),$$

$$U(i,k,l):=\begin{cases}1&\parbox{227.62204pt}{if there exist $0\leq k^{\prime}\leq k$ and $0\leq l^{\prime}\leq l-1$ such that $U(i-1,k^{\prime},l^{\prime})=\mu_{\xi}(u_{i},k-k^{\prime},l-1-l^{\prime})=1,$}\\ 0&\text{o.w.}\end{cases}$$

$$\begin{array}{rl}{\varphi }_T(A_1)\le \xi & \iff c(\partial A_1)\le \xi \omega (A_1)\iff c(e_u)+\sum _{e\in F_{\mathcal{A}}}c(e)\le \xi \left(\omega (T_u)-\sum _{e\in F_{\mathcal{A}}}\omega (T_e)\right)\\ & \iff {\gamma }_{\xi }(\mathcal{A})\le \xi \omega (T_u)-c(e_u).\end{array}$$

$$X_{\xi }(i,k,l):=\{\begin{array}{ll}\min \{{\epsilon }_{\xi }(u{u}_i),{\Gamma }_{\xi }(u_i,k,l)\}& \text{if }{\mu }_{\xi }(u_i,k-1,l)=1,\\ {\Gamma }_{\xi }(u_i,k,l)& \text{o.w.}\end{array}$$

$$Y_{\xi }(1,k,l):=X_{\xi }(1,k,l),$$

$$Y_{\xi }(i,k,l):=\min \{Y_{\xi }(i-1,k^{\prime },l^{\prime })+X_{\xi }(i,k+1-k^{\prime },l-l^{\prime }):1\le k^{\prime }\le k,0\le l^{\prime }\le l\}.$$

$${\Gamma }_{\xi }(u,k,l)=Y_{\xi }(d,k,l).$$

$$\Gamma (k,l)=\min \{{\Gamma }^{\prime }(k^{\prime },l^{\prime })+{\Gamma }^{\prime \prime }(k^{\prime \prime },l^{\prime \prime }):k^{\prime }+k^{\prime \prime }=k+1,l^{\prime }+l^{\prime \prime }=l\}.$$

$${\varphi }_T(\mathcal{A})=\underset{1\le i\le k}{\max }\left\{{\varphi }_T(A_i)=\frac{c(\partial A_i)+p(A_i)}{\omega (A_i)}\right\}\le \xi$$

$${\epsilon }_{\xi }(e):=\xi \omega (T_e)+c(e)-p(T_e).$$

$$Z_{\xi }(1,k,l)={\mu }_{\xi }(r_1,k,l).$$

$$Z(i,k,l):=\{\begin{array}{ll}1& \text{if there exists }0\le k^{\prime }\le k\text{ and }0\le l^{\prime }\le l,\text{ s.t. }\\ & Z(i-1,k^{\prime },l^{\prime })={\mu }_{\xi }(r_i,k-k^{\prime },l-l^{\prime })=1,\\ 0& \text{o.w.}\end{array}$$

$$p(v):=\sum _{e\in E(\{v\},S_1)}c(e).$$

$${\mu }_{\xi }(u,k,l)=\{\begin{array}{ll}1& \text{if u∈S and Γξ(u,k,l)≤ξω(Tu)−c(eu),}\\ 1& \text{if u∈S and either Γξ(u,k,l)≤ξω(Tu)−c(eu) or U(d,k,l)=1,}\\ 0& \text{o.w.}\end{array}$$

Full text

• Key words: sparsest cut problem, isoperimetric number, Cheeger constant, normalized cut, graph partitioning, computational complexity, weighted trees.
• Subject classification: 05C85, 68Q25, 68R10.

1 Introduction

Data clustering is definitely among the main topics of modern computer science with an indispensable role in data mining, image and signal processing, network and data analysis, and data summarization (e.g. see [13] and references therein). Considering the current status of data science, one may name some fundamental challenges in this field, among many others, as follows:

• •

Clustering huge and usually high-dimensional data.

• •

Clustering in presence of outliers and anomalies.

• •

Clustering non-geometric (usually non-Euclidean) data.

• •

Clustering with no prior information about the number of clusters or other features of data (as model of the source etc.).

Needless to say, in each case, efficiency and time-complexity of the proposed algorithms are global parameters with a decisive role in applicability.

The subject of this article falls into the setup of clustering in an unsupervised and static graph-based data presentation. It is instructive to note that the graph-based approach essentially provides data presentation in a very general (not necessarily Euclidean) setting in terms of similarity kernels. In this respect, one of the main well-studied criteria is the “sparsest cut problem” which apart from tremendous real-world applications in the context of spectral clustering (see e.g. [21, 19]), has played a crucial role in the development of many subjects in theoretical computer science (see e.g. [22, 7]).

Our main objective in this article is to improve this approach, which is essentially based on solving a suitable subpartitioning problem on a corresponding minimum spanning tree, by providing an algorithm that not only gives rise to a fast clustering procedure, but also provides good control on determining the number of clusters and outliers. The procedure is based on a dynamic programming which runs in the worst case in $O(k^{2}n^{3})$, where $n$ is the data size and $k$ is the number of clusters. Also, the algorithm runs in linear time in terms of the data size when the number of clusters $k$ and the upper bound on the number of outliers are both constant (which is the case in the most prevalent applications). To the best of our knowledge, the partitioning problem solved by the proposed algorithm (Algorithm 3) is among the most challenging problems in this literature which is efficiently solvable, while we will also dwell on some important consequences in what follows.

1.1 A formal setup and the main result

Partitioning problems are essentially as old as graph theory itself, with wide applications in science and technology. In particular, one may refer to the unnormalized partitioning problems that usually are considered as different versions of minimum cut problems as well as the normalized versions which are more plausible in real applications, however, are much harder to resolve. One of the main problems in the category of normalized cut criteria is the sparsest cut problem which is defined as follows. Given a graph $G$, the sparsest cut problem asks for a cut (a subset of vertices) which has the minimum edge expansion, i.e.

[TABLE]

where $\overline{S}:=V(G)\setminus S$ and $\partial S$ is the set of all edges with exactly one end in $S$. The sparsest cut problem is known to be an $NP-$hard problem on general graphs [21, 18]. Efforts to find an efficient algorithm for a good approximation of this problem have triggered off the development of many subfields of computer science and have had a significant influence on algorithm design and complexity theory. It is amazing to see that recent advances in computer science have given rise to a culmination of ideas not only from the classical graph theoretic point of view but also from the more geometric point of view discussed in the theory of Riemannian manifolds and stochastic processes [23]. Up to now, the best known approximation result for the sparsest cut problem is due to Arora, Rao, and Vazirani [3] which gives an $O(\sqrt{\log n})$ approximation algorithm.

It is also worth noting that the invariant defined in (1) has an intimate connection with the second eigenvalue of the associated Laplacian operator. In fact, relaxation of the minimization problem in (1) to the Euclidean norm for real functions (i.e. changing the edge expansion to the Euclidean $2$-norm of the gradient of real functions which is the energy representable by the Laplacian operator) gives rise to an eigenvalue problem which is efficiently solvable, while estimating the approximation ratio of this relaxation has led to some fundamental contributions (e.g. see [1, 2]). These relations, known as Cheeger’s inequalities, also exert considerable influence over constructing the expander graphs as well as the study of the mixing time of Markov chains (see e.g. [15, 14]). In general, although the motivating problems in these fields of study are usually different, the synergistic effect of methods and techniques have flourished into one of the most active and productive topics in mathematics and computer science.

Recently, some generalizations of the sparsest cut problem have been studied in the literature. Here, we study a generalization which extends two-way partitioning into $k-$way connected subpartitioning and allows some vertices to lie outside the parts.

To formulate the problem precisely, let us first fix our notation and terminology. We assume that the data is given as a simple and finite weighted graph $G=(V,E,\omega,c)$ in which $\omega:V\to{\mathbb{Q}}^{+}$ and $c:E\to{\mathbb{Q}}^{+}$ are the vertex and edge weight functions, respectively. Note that in the literature close to applications the function $c$ is sometimes referred to as the kernel or the similarity, while from a geometric point of view the graph can also be considered as a discrete metric-measure space, where the distance function is usually chosen to be proportional to some inverse function of $c$. In this setting, by an unweighted graph we mean a graph in which all the vertex and edge weights are equal to $1$.

Given a graph $G=(V,E,\omega,c)$ and a subset of vertices $S\subseteq V$, the edge exapnsion or the conductance of $S$, is defined as

[TABLE]

where,

[TABLE]

From a geometric point of view, the conductance can be interpreted as a normalized norm of a gradient function or a normalized energy (e.g. see [5, 6] for more on the geometric interpretations). The set ${\mathscr{D}}_{k}(V)$ is defined to be the set of all $k$-subpartitions $\{A_{{1}},\ldots,A_{{k}}\}:=\{A_{{i}}\}^{k}_{1}$ of $V$, in which $A_{{i}}$’s are nonempty disjoint subsets of $V$. The residue of a subpartition $\{A_{{1}},\ldots,A_{{k}}\}$ is defined to be the set $R:=V-\cup_{{i=1}}^{{k}}A_{{i}}$. The set of all $k$-partitions of $V$, which is denoted by ${\mathscr{P}}_{{k}}(V)$, is the subclass of ${\mathscr{D}}_{{k}}(V)$ containing all $k$-subpartitions $\{A_{{i}}\}^{k}_{1}$ for which $\cup_{{i=1}}^{{k}}A_{{i}}=V$ (i.e. $R=\emptyset$). A subpartition (or a partition in particular) is said to be connected if the subgraph induced on each of its parts is a connected subgraph of $G$. A generalization of the sparsest cut problem can be formulated as follows.

Definition 1**.**

Given a weighted graph $G=(V,E,\omega,c)$ and a positive integer $k$, $1\leq k\leq|V|$, the $k$th isoperimetric number is defined as,

[TABLE]

Furthermore, considering the partitions, the $k$th minimum normalized cut number is defined as,

[TABLE]

A vertex $v\in V$ is called a $k$-outlier, if there exists a minimizing subpartition achieving $\iota_{k}(G)$, while $v$ lies in its residue. It is well-known that $\iota_{2}=\tilde{\iota}_{2}$ (see [8]) and the common value is usually called the Cheeger constant or edge expansion in the literature. $\blacktriangle$

In this regard, Louis et al. in [17] provide a polynomial time approximation algorithm which outputs a $(1-\epsilon)k$-partition of the vertex set such that each piece has expansion at most $O_{\epsilon}(\sqrt{\log n\log k})$ times $\tilde{\iota}_{{k}}(G)$ (for every positive number $\epsilon$). Also, in [16], higher-order Cheeger’s inequalities have been proved which relate the above parameters to the eigenvalues of the associated Laplacian Matrix (see also [8, 10]).

Prior to formulating our problem, let us discuss some facts. First, one may note that as an imprecise rule of thumb, changing the cost function of a partitioning problem, from the normalized form to the unnormalized form, from partitions to subpartitions, or from the mean (i.e. $1$-norm) to the max (i.e. $\infty$-norm) generally makes the problem more tractable in the sense that finding more efficient algorithms to solve the problem become more probable. One of our major observations in this article is the fact that the restriction of the search space to “connected” subpartitions reduces the complexity of the problem too. In particular, this distinction is much comprehensible when the graph is a tree where the restriction on subpartitions to be connected reduces the complexity of the problem from $NP-$hard to polynomial time. Also, note that this restriction is to the best of our advantage in the sense that a cluster is more expected to be represented by a connected subgraph than a disconnected one (based on intra-similarity of the objects within a cluster). Hence, as far as clustering is concerned, this can be considered as an acceptable assumption. As a matter of fact, in what follows, we show that such a change to the better will give rise to an efficient algorithm for clustering with a control on the number of parts and outliers.

We denote the main problem, i.e. the multi-way sparsest cut problem with a control on the residue number, by the acronym “MSC problem” which is defined as follows. **MSC Problem.

**

INSTANCE:

A weighted graph $G=(V,E,\omega,c)$, nonnegative integers $\kappa\in\mathbb{Z}^{+}$ and $\lambda\in\mathbb{Z}^{+}$ and a positive rational number $\xi\in{\mathbb{Q}}^{+}$.

QUERY:

Does there exist a $\kappa$-subpartition of $V$ such as $\{A_{{i}}\}^{\kappa}_{1}\in\mathscr{D}_{\kappa}(V)$ such that $\displaystyle{\max_{1\leq i\leq\kappa}}\{\phi_{G}(A_{{i}})\}\leq\xi$ and its residue number is at most $\lambda$, i.e. $|V\setminus\cup_{i=1}^{\kappa}A_{i}|\leq\lambda$?

The MSC problem is known to be a hard problem even when the graph is of its simplest form, i.e. a tree. When the graph $G$ is a tree, it is proved in [9] that MSC problem is $NP-$complete even when the tree is unweighted and $\lambda$ is constant (e.g. $\lambda=0$). Nonetheless, it is shown there that the problem is solvable in linear time for weighted trees when we drop the restriction on the residue number (i.e. $\lambda=|V|$). An improvement of this result has effectively been applied to real clustering problems for large data-sets [11].

The main contribution of this article (Algorithm 3) is to show that although MSC problem is $NP-$complete for trees, it becomes tractable when the search space is restricted to connected subpartitions. In other words, the following problem abbreviated by CMSC can be solved in polynomial time for weighted trees. **CMSC Problem.

**

INSTANCE:

A weighted graph $G=(V,E,\omega,c)$, nonnegative integers $\kappa\in\mathbb{Z}^{+}$ and $\lambda\in\mathbb{Z}^{+}$ and a positive rational number $\xi\in{\mathbb{Q}}^{+}$.

QUERY:

Does there exist a connected $\kappa$-subpartition of $V$ such as $\{A_{{i}}\}^{\kappa}_{1}\in\mathscr{D}_{\kappa}(V)$ such that $\displaystyle{\max_{1\leq i\leq\kappa}}\{\phi_{G}(A_{{i}})\}\leq\xi$ and its residue number is at most $\lambda$, i.e. $|V\setminus\cup_{i=1}^{\kappa}A_{i}|\leq\lambda$?

This result along with the fact that the minimum spanning tree of a geometric metric-measure space inherits a large part of the geometry of the space, can lead to a good approximation for MSC problem for general graphs. This can justify the importance of the problem on weighted trees when applications are concerned. Let us consider some consequences of this result.

Firstly, note that given a weighted tree $T$ and integers $\kappa$ and $\lambda$, finding the minimum number $\xi$ for which there exists a connected $\kappa-$subpartition with the residue number at most $\lambda$ and $\max_{1\leq i\leq\kappa}\{\phi_{G}(A_{i})\}\leq\xi$ (as well as finding the minimizing subpartition) can be done in polynomial time by applying our algorithm iteratively along with a simple binary search.

Secondly, given a weighted tree $T$ and numbers $\xi,\lambda$ (the worst edge expansion of the clusters), we can obtain a number $k_{{max}}(T,\xi)$, denoting the maximum number of parts for which the answer to CMSC problem is positive. This by itself is an important piece of information when one considers the large existing literature discussing how to determine the number of clusters for a clustering algorithm (e.g. see [20] for $k$-means).

Thirdly, from another point of view, CMSC problem can be considered as a problem of outlier-robust clustering where a solution will provide information on the number of outliers. It is well-known that detection of outliers and anomalies in data-sets are among the most challenging problems in the field, not just because of the hardness of the problem itself, but since the concepts themselves are quite fuzzy and depend on many different parameters as scaling or distribution of the source (e.g. see [4, 12] for the background). These facts, and in particular, lack of a universal sound and precise definition, is among the first obstacles when one is dealing with these kinds of problems. In [11] some evidence has been discussed that how the data remained in the exterior of the clusters in MSC problem can be justified to be actual outliers in some sense.

Finally, our method can be extended to handle some more general semi-supervised settings where a number of training samples are given by the user which are forced or forbidden to lie in outliers (see Section 4).

The organization of forthcoming sections is as follows. In Section 2, we give required definitions and notations as well as the lemmas which justify our algorithm. In Section 3, we present the main algorithm and explain how it can find the optimal subpartition. We also compute the time complexity of our algorithm. Finally, in Section 4, we discuss some extensions which handle more realistic models.

2 Preliminaries

Let $T$ be a rooted tree with root $r$. There is a natural partial order induced through the root on the vertices and edges of $T$ defined as $u\leq v$ for two vertices $u$ and $v$ whenever there is a path $P(r,v,u)$ in $T$ starting from $r$ and ending at $u$ which contains $v$. Similarly, $e\leq e^{\prime}$ for two edges $e$ and $e^{\prime}$ whenever there is a path $P(r,e^{\prime},e)$ in $T$ starting from $r$ and containing $e$ and $e^{\prime}$ such that $e^{\prime}$ is closer than $e$ to $r$ on $P$. In this setting, note that for any $u\not=r$ there exists a unique minimal vertex $v$, with $v\geq u$ and an edge $e_{{u}}:=uv$, where $v$ and $e_{u}$ are called the parent vertex and the parent edge of $u$, respectively (and also $u$ is called the child of $v$). Also, for a given edge $e=uv$ with $u\leq v$ we may refer to $e^{-}=u$ and $e^{+}=v$, intermittently. For some technical reasons, we add one new vertex $r^{\prime}$ to $T$ and connect it to $r$ and define the parent edge of $r$, $e_{r}$, as the edge $rr^{\prime}$. Also, we set $\omega(r^{\prime})=c(e_{r})=0$.

If $F$ is a subset of edges of $T$, then $M(F)$ is the set of maximal elements of $F$ with respect to the natural partial order of $T$. Given a vertex $u$ with the parent edge $e_{u}$, the subtree $T_{u}=T_{e_{u}}$ refers to the subtree induced on the set $\{v\in V(T):v\leq u\}$. Therefore, $T_{r}=T_{e_{r}}=T$.

Let $T=(V,E,\omega,c)$ be a weighted tree and $\xi$ be a fixed positive number. For every integer $0\leq k\leq|V|$, define $\mathscr{C}_{{k}}(T)$ to be the class of all $k$-subpartitions $\mathcal{A}=\{A_{i}\}_{1}^{k}$ such that for each $1\leq i\leq k$, $A_{i}\subseteq V(T)$ and the induced subgraph of $T$ on $A_{i}$ is connected (i.e. $A_{i}$ is a subtree of $T$). Also, given a subpartition $\mathcal{A}=\{A_{i}\}_{1}^{k}\in\mathscr{C}_{k}(T)$, its residue set is defined as $R(\mathcal{A},T):=V(T)\setminus\cup_{i=1}^{k}A_{i}$. We also define,

[TABLE]

In the following we describe the idea that our algorithm is based on and also prove the correctness of the algorithm. First, note that since we are looking for subsets with small edge expansion, when we cut an edge $e$, the subset containing $e^{+}$ sustains a loss in its edge expansion. The cause of this deficiency is that the numerator of the edge expansion is added by $c(e)$ and the denominator is subtracted by $\omega(T_{e})$. With this intuition, for every edge $e\in E$, define

[TABLE]

Now, let $\kappa$ and $\lambda$ be two nonnegative integers and for every integers $1\leq k\leq\kappa$, $0\leq l\leq\lambda$ and vertex $u\in V(T)$, define $\mathscr{C}_{\xi}(u,k,l)$ to be the set of all $k$-subpartitions $\mathcal{A}=\{A_{i}\}_{1}^{k}$ in $\mathscr{C}_{k}(T_{u})$ such that $u\in A_{1}$ and $|R(\mathcal{A},T_{u})|\leq l$ and for each $2\leq i\leq k$, we have $\phi_{T}(A_{{i}})\leq\xi$. For each such subpartition $\mathcal{A}$, let $F_{\mathcal{A}}:=\partial A_{1}\setminus\{e_{u}\}$. Note that any pair of edges in $F_{\mathcal{A}}$ are incomparable and define,

[TABLE]

We will shortly see that minimizing the edge expansion $\phi_{T}(A_{1})$, in some sense, is equivalent to minimizing $\gamma_{\xi}(\mathcal{A})$ (see (6)). Thus, define,

[TABLE]

On the other hand, for every integers $0\leq k\leq\kappa$ and $0\leq l\leq\lambda$ and vertex $u\in V(T)$, define $\mu_{\xi}(u,k,l)$ to be equal to $1$ if there exists a connected $k$-subpartition $\mathcal{A}=\{A_{i}\}_{1}^{k}\in\mathscr{C}_{k}(T_{u})$ such that $\phi_{T}(\mathcal{A})\leq\xi$ and $|R(\mathcal{A},T_{u})|\leq l$ and it is equal to [math], otherwise. Note that, although $A_{i}$’s are subsets of $V(T_{u})$, $\phi_{T}(A_{i})$ is computed in the whole tree $T$. Also, note that for every vertex $u\in V(T)$ and integer $l$, we have

[TABLE]

In fact, our main goal is to compute the parameter $\mu_{\xi}(r,\kappa,\lambda)$, since evidently the answer to CMSC problem is yes if and only if $\mu_{\xi}(r,\kappa,\lambda)=1$. In the sequel, we are going to show that the parameters $\Gamma_{\xi}(u,k,l)$ and $\mu_{\xi}(u,k,l)$ can be computed recursively in a breath-first scanning of vertices towards the root. First, in the following, we explain how one can compute $\mu_{\xi}(u,k,l)$ recursively in terms of the values $\Gamma_{\xi}(u,k,l)$, $1\leq k\leq\kappa,0\leq l\leq\lambda$. For this, let $\xi,\kappa,\lambda$ be fixed and given a vertex $u$, let $(u_{1},\ldots,u_{d})$ be an ordering of all of its children. Now, for every integers $0\leq k\leq\kappa$, $0\leq l\leq\lambda$, define

[TABLE]

and for every $2\leq i\leq d$, define

[TABLE]

In the following lemma, we show how one can use the recursion in (5) to compute the function $\mu_{\xi}$.

Lemma 2**.**

Let $u$ be a vertex in a rooted tree $T$, $\xi\geq 0$ be a number and $\lambda\geq 0,\kappa\geq 1$ be two integers. Also, let $u_{1},\ldots,u_{d}$ be the children of $u$ in $T$. For every integers $0\leq k\leq\kappa$, $0\leq l\leq\lambda$, $\mu_{\xi}(u,k,l)=1$ if and only if either $\Gamma_{\xi}(u,k,l)\leq\xi\,\omega(T_{u})-c(e_{u})$, or $U(d,k,l)=1$.

Proof.

Suppose that $\mu_{\xi}(u,k,l)=1$ and let $\mathcal{A}=\{A_{i}\}_{1}^{k}\in\mathscr{C}_{k}(T_{u})$ be a connected subpartition where $R(\mathcal{A},T_{u})\leq l$ and $\phi_{T}(\mathcal{A})\leq\xi$. First, assume that $u\in R(\mathcal{A},T_{u})$. Thus, $\mathcal{A}$ itself can be partitioned into $d$ connected subpartitions $\mathcal{A}_{1},\ldots,\mathcal{A}_{d}$ such that $\mathcal{A}_{i}\in\mathscr{C}_{k_{i}}(T_{u_{i}})$, for some integers $k_{i}$, where $k_{1}+\cdots+k_{d}=k$. Also, let $l_{i}=|R(\mathcal{A}_{i},T_{u_{i}})|$. Therefore, by definition $\mu_{\xi}(u_{i},k_{i},l_{i})=1$ and $l_{1}+\cdots+l_{d}=|R(\mathcal{A},T_{u})\setminus\{u\}|\leq l-1$. Thus, again by definition $U(d,k,l)=1$. Next, suppose that $u\not\in R(\mathcal{A},T_{u})$ and so, without loss of generality, assume that $u\in A_{1}$. Then,

[TABLE]

Therefore, $\Gamma_{\xi}(u,k,l)\leq\xi\,\omega(T_{u})-c(e_{u})$. This implies that if $\mu_{\xi}(u,k,l)=1$, then either $\Gamma_{\xi}(u,k,l)\leq\xi\,\omega(T_{u})-c(e_{u})$, or $U(d,k,l)=1$.

Now, suppose that $U(d,k,l)=1$. Then, there exist integers $l_{1},\ldots,l_{d}$ and $k_{1},\ldots,k_{d}$ such that $\sum_{i=1}^{d}l_{i}=l-1$, $\sum_{i=1}^{d}k_{i}=k$ and $\mu_{\xi}(u_{i},k_{i},l_{i})=1$, for all $1\leq i\leq d$. Thus, for each $1\leq i\leq d$, there exists $\mathcal{A}_{i}\in\mathscr{C}_{k_{i}}(T_{u_{i}})$ such that $R(\mathcal{A}_{i},T_{u_{i}})\leq l_{i}$ and $\phi_{T}(\mathcal{A}_{i})\leq\xi$. Define $\mathcal{A}=\cup_{i=1}^{d}\mathcal{A}_{i}$. Thus, $\mathcal{A}\in\mathscr{C}_{k}(T_{u})$ and $|R(\mathcal{A},T_{u})|\leq l$. Hence, $\mu_{\xi}(u,k,l)=1$.

Finally, suppose that $\Gamma_{\xi}(u,k,l)\leq\xi\,\omega(T_{u})-c(e_{u})$. Also, let $\mathcal{A}\in\mathscr{C}_{\xi}(u,k,l)$ be a minimizer with $\gamma_{\xi}(\mathcal{A})=\Gamma_{\xi}(u,k,l)$. Then, by definition, for every $2\leq i\leq k$, $\phi_{T}(A_{i})\leq\xi$ and $|R(\mathcal{A},T_{u})|\leq l$ and by (6), $\phi_{T}(A_{1})\leq\xi$. Hence, $\mu_{\xi}(u,k,l)=1$. This completes the proof. ∎

As we see in Lemma 2, in order to obtain the value of $\mu_{\xi}(u,k,l)$, we require to have the value of $\Gamma_{\xi}(u,k,l)$. In the next step, we show that given $\xi,\kappa$ and $\lambda$, how one may compute $\Gamma_{\xi}(u,k,l)$ efficiently for all vertices $u$ and integers $1\leq k\leq\kappa$, $0\leq l\leq\lambda$. For this, let $\xi,\kappa,\lambda$ be fixed and given a vertex $u$, let $(u_{1},\ldots,u_{d})$ be an ordering of all of its children. Now, for every integers $1\leq k\leq\kappa$, $0\leq l\leq\lambda$ and $1\leq i\leq d$, define

[TABLE]

Also, define

[TABLE]

and for every $2\leq i\leq d$, define

[TABLE]

The following lemma shows how to compute the function $\Gamma_{\xi}$ using recursion (9).

Lemma 3**.**

Let $T$ be a rooted tree, $\xi\geq 0$ be a number and $\lambda\geq 0,\kappa\geq 1$ be two integers. Then, for every vertex $u\in V(T)$ with $d$ children $(u_{1},\ldots,u_{d})$ and every integers $1\leq k\leq\kappa$ and $0\leq l\leq\lambda$, we have

[TABLE]

Proof.

We prove the lemma by induction on the number $d$. Let $\mathcal{A}=\{A_{i}\}_{1}^{k}\in\mathscr{C}_{\xi}(u,k,l)$ be a $k$-subpartition. First, suppose that $d=1$. If $uu_{1}\in F_{\mathcal{A}}$, then $\gamma_{\xi}(\mathcal{A})=\varepsilon_{\xi}(uu_{1})$ and $A_{2},\ldots,A_{k}\subseteq V(T_{u_{1}})$, so $\mu_{\xi}(u_{1},k-1,l)=1$. Also, if $uu_{1}\not\in F_{\mathcal{A}}$, then $u_{1}\in A_{1}$ and $\gamma_{\xi}(\mathcal{A})\geq\Gamma_{\xi}(u_{1},k,l)$. Therefore, $\Gamma_{\xi}(u,k,l)=X_{\xi}(1,k,l)=Y_{\xi}(1,k,l)$ as in (7) and (8).

Now, suppose that $d\geq 2$. Let $T^{\prime}:=T\setminus T_{u_{d}}$ and $T^{\prime\prime}:=T\setminus(\cup_{i=1}^{d-1}T_{u_{i}})$ and $\Gamma(k,l)$, $\Gamma^{\prime}(k,l)$ and $\Gamma^{\prime\prime}(k,l)$ be the values of $\Gamma_{\xi}(u,k,l)$ for the trees $T$, $T^{\prime}$ and $T^{\prime\prime}$, respectively. Also, let $l^{\prime}=|R(\mathcal{A},T)\cap V(T^{\prime})|$, $l^{\prime\prime}=l-l^{\prime}$ and let $k^{\prime}$ (resp. $k^{\prime\prime}$) be the number of sets $A_{i}$ which intersect $V(T^{\prime})$ (resp. $V(T^{\prime\prime})$). Then, evidently we have $k^{\prime}+k^{\prime\prime}=k+1$ (note that $A_{1}$ intersects both $V(T^{\prime})$ and $V(T^{\prime\prime})$) and $\gamma_{\xi}(\mathcal{A})\geq\Gamma^{\prime}(k^{\prime},l^{\prime})+\Gamma^{\prime\prime}(k^{\prime\prime},l^{\prime\prime})$. Therefore,

[TABLE]

On the other hand, by the induction hypothesis, we have $\Gamma^{\prime}(k^{\prime},l^{\prime})=Y_{\xi}(d-1,k^{\prime},l^{\prime})$ and $\Gamma^{\prime\prime}(k^{\prime\prime},l^{\prime\prime})=X_{\xi}(d,k^{\prime\prime},l^{\prime\prime})$. Hence, by (9), we have $\Gamma(k,l)=Y_{\xi}(d,k,l)$ and we are done. ∎

3 The algorithm

In this section, using Lemmas 2 and 3, we provide an algorithm to solve the CMSC problem for all weighted trees. The cores of the algorithm are two dynamic programmings. The final solution to the problem is given in Algorithm 3 which scans the vertices in a BFS order towards the root $r$ and computes recursively the values of $\Gamma_{\xi}(u,k,l)$ and $\mu_{\xi}(u,k,l)$, for $1\leq k\leq\kappa$ and $0\leq l\leq\lambda$. The structure of Algorithm 3 which deploys Algorithms 1 and 2 as two subroutines, is as follows.

First, for all leaves $u$ (vertices with no children), it computes the values of $\Gamma_{\xi}(u,k,l)$ and $\mu_{\xi}(u,k,l)$ (Lines 8-13 in Algorithm 3). Next, for a vertex $u$, with children $(u_{1},\ldots,u_{d})$, according to Lemma 3, it applies a dynamic programming (Algorithm 1) based on the recursion given in Equations (8) and (9), to obtain the value of $\Gamma_{\xi}(u,k,l)$, assuming the values of $\mu_{\xi}(u_{i},k,l)$ and $\Gamma_{\xi}(u_{i},k,l)$ are given. Finally, according to Lemma 2, it applies another dynamic programming (Algorithm 2) based on the recursion given in (5) to obtain the value of $\mu_{\xi}(u,k,l)$, assuming the values of $\Gamma_{\xi}(u,k,l)$ and $\mu_{\xi}(u_{i},k,l)$ are given. The backtracking ends up outputting the value of $\mu_{\xi}(r,\kappa,\lambda)$ which is equal to $1$ if and only if there exists a connected $\kappa$-subpartition $\mathcal{A}$ with $\phi_{T}(\mathcal{A})\leq\xi$ and $|R(\mathcal{A},T)|\leq\lambda$. This completes the solution.

3.1 Time complexity

The time complexity of the provided algorithms can be computed as follows. In Algorithm 1, Lines 2-12 can be done in $O(d(\lambda+1)\kappa)$. Also, Lines 14-25 can be performed in $O(d(\lambda+1)^{2}\kappa^{2})$. In Algorithm 2, Lines 2-9 run in $O((\lambda+1)\kappa)$ and Lines 11-26 run in $O(d(\lambda+1)^{2}\kappa^{2})$. Hence, the runtime of Algorithm 3 is in $O((\lambda+1)^{2}\kappa^{2}n)$. Since in real applications, the values of $\kappa$ and $\lambda$ are mostly much smaller than $n$, we can assume that the algorithm runs in linear time with respect to the number of nodes.

3.2 Constructing the optimal subpartition

Now, we show that during the execution of Algorithm 3, how one can construct a subpartition $\mathcal{A}\in\mathscr{C}_{\kappa}(T)$ with $\phi_{T}(\mathcal{A})\leq\xi$ and $|R(\mathcal{A},T)|\leq\lambda$ (if there exists). Let $\xi,\kappa$ and $\lambda$ be fixed and for every vertex $u\in V(T)$ and $1\leq k\leq\kappa$ and $0\leq l\leq\lambda$, if $\mu_{\xi}(u,k,l)=1$, then let $\mathcal{A}_{\mu}=\mathcal{A}_{\mu}(u,k,l)$ be a $k-$subpartition in $\mathscr{C}_{k}(T_{u})$ such that $\phi_{T}(\mathcal{A}_{\mu})\leq\xi$ and $|R(\mathcal{A}_{\mu},T_{u})|\leq l$. Also, if $\mu_{\xi}(u,k,l)=0$, let $\mathcal{A}_{\mu}=\mathcal{A}_{\mu}(u,k,l):=\emptyset$. Then, the subpartition $\mathcal{A}_{\mu}(r,\kappa,\lambda)$ is what we are looking for. Also, let $\mathcal{A}_{\Gamma}=\mathcal{A}_{\Gamma}(u,k,l)$ be a subpartition in $\mathscr{C}_{\xi}(u,k,l)$ which minimizes (3).

Now, let $u$ be a vertex with children $(u_{1},\ldots,u_{d})$. First, according to Algorithm 2 and assuming that we have all the subpartitions $\mathcal{A}_{\Gamma}(u,k,l)$ and $\mathcal{A}_{\mu}(u_{i},k,l)$, we explain how to obtain $\mathcal{A}_{\mu}(u,k,l)$. For this, throughout the execution of Algorithm 2, in Line 5, if $\Gamma_{\xi}(u,k,l)\leq\xi\omega(T_{u})-c(e_{u})$, then set $\mathcal{A}_{\mu}(u,k,l):=\mathcal{A}_{\Gamma}(u,k,l)$, otherwise set $\mathcal{A}_{\mu}(u,k,l):=\emptyset$. Also, in Line 10, if $\mu_{\xi}(u_{1},k,l)=1$, then set $\mathcal{U}(k,l):=\mathcal{A}_{\mu}(u_{1},k,l)$ and in Line 18, if $U(i-1,k^{\prime},l^{\prime})=\mu_{\xi}(u_{i},k-k^{\prime},l-1-l^{\prime})=1$, then set $\mathcal{U}(k,l):=\mathcal{U}(k^{\prime},l^{\prime})\cup\mathcal{A}_{\mu}(u_{i},k-k^{\prime},l-1-l^{\prime})$. Finally, in Line 27, if $U(d,k,l)=1$, then set $\mathcal{A}_{\mu}(u,k,l):=\mathcal{U}(k,l)$.

Next, according to Algorithm 1 and assuming that we have all the subpartitions $\mathcal{A}_{\mu}(u_{i},k,l)$ and $\mathcal{A}_{\Gamma}(u_{i},k,l)$, we explain how to obtain $\mathcal{A}_{\Gamma}(u,k,l)$. First, throughout the execution of Algorithm 1, in Line 5, if $\mu_{\xi}(u_{i},k-1,l)=1$ and $\varepsilon_{i}\leq\Gamma_{\xi}(u_{i},k,l)$, then set $\mathcal{X}(i,k,l):=\mathcal{A}_{\mu}(u_{i},k-1,l)\cup\{\{u\}\}$, otherwise let $\mathcal{X}(i,k,l)$ be the subpartition obtained from $\mathcal{A}_{\Gamma}(u_{i},k,l)$ by adding the vertex $u$ to the set containing $u_{1}$. Also, in Line 13, set $\mathcal{Y}(k,l):=\mathcal{X}(1,k,l)$. Next, in Line 20, if $Y(i-1,k^{\prime},l^{\prime})+X(i,k+1-k^{\prime},l-l^{\prime})\leq Y(i,k,l)$, then let $\mathcal{Y}(k,l)$ be obtained from the disjoint union of $\mathcal{Y}(k^{\prime},l^{\prime})$ and $\mathcal{X}(i,k+1-k^{\prime},l-l^{\prime})$ by merging two sets containing the vertex $u$. Finally, in Line 26, set $\mathcal{A}_{\Gamma}(u,k,l):=\mathcal{Y}(k,l)$.

4 Towards more extensions

In this section, we show that our presented scheme can be generalized to solve the following more realizable problems efficiently:

Solving CMSC problem on trees with potentials. 2. 2.

Solving CMSC problem on forests. 3. 3.

Solving the following semi-supervised problem: Given a weighted graph $G=(V,E,\omega,c)$ (not necessarily a forest), two disjoint subsets $S_{1},S_{2}\subseteq V$, rational number $\xi$ and integers $\kappa,\lambda$, such that the induced subgraph of $G$ on $V(G)\setminus S_{1}$ is a forest. Does there exist a connected subpartition $\mathcal{A}\in\mathscr{D}_{\kappa}(V)$ such that $\phi_{G}(\mathcal{A})\leq\xi$, $|R(\mathcal{A},G)|\leq\lambda$, $S_{1}\subseteq R(\mathcal{A},G)$ and $S_{2}\cap R(\mathcal{A},G)=\emptyset$?

In the following, we elaborate on the modifications that should be made to tackle the above settings.

In the setting of trees with potentials, each vertex $v\in V(T)$ is endowed with a potential weight, say $p(v)$, which is a nonnegative number and the goal is to determine whether there exists a connected $k$-subpartition $\mathcal{A}=\{A_{{i}}\}^{k}_{1}\in\mathscr{D}_{k}(V)$ such that

[TABLE]

and $|R(\mathcal{A},T)|\leq\lambda$. We can extend our method to solve this problem using Algorithm 3. First, for each edge $e\in E(T)$, amend the definition of $\varepsilon_{\xi}(e)$ in (2) as follows

[TABLE]

Also, define the functions $\mu_{\xi}$ and $\Gamma_{\xi}$ analogously. Next, with a similar argument as in Lemma 2, one may prove that $\mu_{\xi}(u,k,l)=1$ if and only if either $\Gamma_{\xi}(u,k,l)\leq\xi\,\omega(T_{u})-c(e_{u})-p(T_{u})$, or $U(d,k,l)=1$. Moreover, Lemma 3 is still valid. So, we should just change Line 1 in Algorithm 1 and Line 5 in Algorithm 2, accordingly and then Algorithm 3 works for the new setting. 2. 2.

Suppose that the forest $F$ consists of $c$ disjoint trees $T_{1},\ldots,T_{c}$ rooted at $r_{1},\ldots,r_{c}$ respectively. Also, let $\xi,\kappa,\lambda$ be fixed. First, using Algorithm 3, compute the value of $\mu_{\xi}(r_{i},k,l)$, for every integers $0\leq l\leq\lambda$, $0\leq k\leq\kappa$ and $1\leq i\leq c$. Also, define

[TABLE]

The following recursion helps us to solve the problem on $F$. For every $2\leq i\leq c$, define

[TABLE]

Then, the solution to CMSC problem is yes if and only if $Z_{\xi}(c,\kappa,\lambda)=1$. Furhermore, One may easily extend this recursion to solve the corresponding problem on forests with potentials. 3. 3.

In this setting some vertices should be or should not be in the residue set. The problem can be solved in the following steps:

First, for each vertex $v\in V(G)$, define a potential as follows

[TABLE]

• -

Now, let $F$ be a forest obtained from $G$ by deleting all vertices in $S_{1}$. Also, let $\lambda^{\prime}:=\lambda-|S_{1}|$.

• -

If $S_{2}$ is empty, then the solution can be obtained by performing the method given in 2 on the forest $F$ with the potential weight $p$ and the numbers $\xi,\kappa,\lambda^{\prime}$. If $S_{2}$ is non-empty, we have to make the following additional modifications to handle the problem.

Suppose that $T$ is a tree and $S\subseteq V(T)$ is a subset of vertices. Also, numbers $\xi,\kappa,\lambda$ are given. We are looking for a connected subpartition $\mathcal{A}\in\mathscr{D}_{\kappa}(V)$ such that $\phi_{T}(\mathcal{A})\leq\xi$, $|R(\mathcal{A},T)|\leq\lambda$ and $S\cap R(\mathcal{A},T)=\emptyset$. Note that Lemma 3 is still valid in this setting. However, in the computation of $\mu_{\xi}(u,k,l)$, for each $u\in V(T)$, in Lemma 2, if $u\in S$, then $u$ is not allowed to be in the residue set. So, the value of $\mu_{\xi}(u,k,l)$ is equal to $1$ if and only if $\Gamma_{\xi}(u,k,l)\leq\xi\,\omega(T_{u})-c(e_{u})$. Thus, with a similar proof as in Lemma 2, we can prove that

[TABLE]

Then, Algorithm 2 can be modified accordingly to compute the value of $\mu_{\xi}(u,k,l)$.

5 Concluding remarks and future work

In this paper, a multi-way sparsest cut problem has been investigated for weighted trees and it was shown that although the problem is $NP-$complete for trees, it becomes tractable when the search space is confined to connected subdomains. One of the strengths of the method is that it has a control on the number of outliers and can manage semi-supervised settings when some data points are forced or forbidden to be outlier. Besides the theoretical importance of the sparsest cut problem, when our method is applied to the minimum spanning tree, it can steer several applications in both unsupervised and semi-supervised clustering.

One may also consider an analogous problem when we are seeking for a subpartition minimizing “the average” (instead of the maximum) of the edge expansions of the parts (e.g. as in [21]). This objective function is more sensitive and exquisite and are more likely to produce high-quality clustering results. Nevertheless, the problem unfortunately turns out to be $NP-$complete on trees even when the search space is restricted to connected subpartitions (or partitions) [9]. Finding a good approximation algorithm for this problem is an interesting and challenging task that can be the purpose of future work in this line of research.

Acknowledgment. We would like to express our sincere thanks to Amir Daneshgar whose valuable comments were crucial in preparing and improving the present article.