Matroids Hitting Sets and Unsupervised Dependency Grammar Induction

TL;DR

This paper introduces a new graph problem related to minimal edge sets that encompass all spanning trees of certain subgraphs, inspired by unsupervised grammar induction, and offers theoretical analysis and algorithms.

Contribution

It formulates a novel graph problem motivated by unsupervised dependency grammar induction and provides reductions, complexity analysis, and an approximation algorithm.

Findings

Problem is computationally complex.

Reduction to known graph problems established.

Approximation algorithm proposed.

Abstract

This paper formulates a novel problem on graphs: find the minimal subset of edges in a fully connected graph, such that the resulting graph contains all spanning trees for a set of specifed sub-graphs. This formulation is motivated by an un-supervised grammar induction problem from computational linguistics. We present a reduction to some known problems and algorithms from graph theory, provide computational complexity results, and describe an approximation algorithm.

Tables

Table 1. Table 1: Greedy algorithm

Input: lexicon $V$, set of $k$ sentences $S_1,\dots ,S_k$ s.t. $S_i\subset V$ ;
Initialize: Assign each pair of words a count of sentences it appears in;
	Sort word pairs (edges) in decreasing order;
	Loop: through edges;
		Add top edge $e$ into the list of edges;
		Adjust edge weights in sentences which contained edge $e$;
	Until each sub-graph has a spanning tree (i.e. sentence has a parse);
Output: a set of spanning trees for all $S_i$;

Equations

$$V=$$

$$E=$$

$$S_{\{i,j\}}=$$

$$S_i^{\prime }=$$

$$OPT(G)\le f(n)\cdot n^3+\left(\genfrac{}{}{0ex}{}{n}{2}\right)$$

$$OPT(G)>\alpha \log n\cdot f(n)\cdot n^3+\left(\genfrac{}{}{0ex}{}{n}{2}\right).$$

$$\frac{\alpha \log n\cdot f(n)\cdot n^3+\left(\genfrac{}{}{0ex}{}{n}{2}\right)}{f(n)\cdot n^3+\left(\genfrac{}{}{0ex}{}{n}{2}\right)}$$

$$\displaystyle\frac{\alpha\log n\cdot\big{(}f(n)\cdot n^{3}+{\textstyle\binom{n}{2}}/(\alpha\log n)\big{)}}{f(n)\cdot n^{3}+{\textstyle\binom{n}{2}}}$$

$$\displaystyle\alpha\log n\cdot\Bigg{(}1-\frac{{\textstyle\binom{n}{2}}\cdot\big{(}1-1/\alpha\log n)\big{)}}{f(n)\cdot n^{3}+{\textstyle\binom{n}{2}}}\Bigg{)}$$

$$OPT(G)\le {\alpha }^{\prime }(|V|-1)+(|E|-|V|+1)$$

$$OPT(G)>{\beta }^{\prime }(|V|-1)+(|E|-|V|+1).$$

$$\frac{{\beta }^{\prime }(|V|-1)+(|E|-|V|+1)}{{\alpha }^{\prime }(|V|-1)+(|E|-|V|+1)}$$

$$1+\frac{({\beta }^{\prime }-{\alpha }^{\prime })(|V|-1)}{{\alpha }^{\prime }(|V|-1)+(|E|-|V|+1)}$$

$$1+\frac{({\beta }^{\prime }-{\alpha }^{\prime })(|V|-1)}{(d/2-1+{\alpha }^{\prime })|V|+1-{\alpha }^{\prime }}$$

$$1+\frac{{\beta }^{\prime }-{\alpha }^{\prime }}{d/2+{\alpha }^{\prime }}\cdot \frac{|V|-1}{|V|-1},$$

$${\mathrm{rank}}_j(S+e)+{\mathrm{rank}}_j(F_i)$$

$${\mathrm{rank}}_j(S+e)-{\mathrm{rank}}_j(S)$$

$$P(S,e)$$

$$s$$

$$\le |\mathcal{O}|\cdot \sum _{i=1}^s\sum _{0\le j<p_i}\frac{1}{R_{i-1}-j}\le |\mathcal{O}|\cdot \log R_0.$$

$$\max \left\{|\bigcup _i{S}_i|:S_i\in {\mathcal{I}}_i^{\ast }\right\}$$

$$\min\left\{\;\lvert\bigcap_{i}\overline{S_{i}}\rvert\,:\,\mbox{$\overline{S_{i}}$ contains a basis for $M_{i}$}\;\right\}$$

$$\min \left\{|\bigcap _i{T}_i|:\text{Ti is a basis for Mi}\right\}$$

Full text

1 - Dept of Computer Science, Univ of British Columbia, Vancouver, Canada

2 - Systems Biology Dept, Harvard Medical School, Boston, USA

3 - Dept of Computer Science, MIT, Boston, USA

4 - Google

∗ - corresponding author peshkin@gmail.com

1 Introduction

Linguistic representations of natural language syntax arrange syntactic dependencies among the words in a sentence into a tree structure, of which the string is a one dimensional projection. We are concerned with the task of analyzing a set of several sentences, looking for the most parsimonious set of corresponding syntactic structures, solely on the basis of co-occurrence of words in sentences. We proceed by first presenting an example, then providing a general formulation of dependency structure and grammar induction.

Consider a sentence ”Her immediate predecessor suffered a nervous breakdown.” A dependency grammar representation of this sentence shown in Figure 1 captures dependency between the subject, the object and the verb, as well as dependency between the determiner and the adjectives and their respective nouns. In this sentence, the subject predecessor and the object breakdown are related to the verb suffered. The verb suffered is the root of the dependency structure, that is illustrated in the diagram by a link to the period. Figure 2 left represents the same dependency structure in a different way by ignoring the direction. Instead the dependence is related to the relative depth in the tree.

In a dependency tree, each word is the mother of its dependents, otherwise known as their head. To linearize the dependency tree in Figure 2.left into a string, we introduce the dependents recursively next to their heads:

iteration 1: suffered

iteration 2: predecessor suffered breakdown

iteration 3: her predecessor suffered a breakdown.

Dependency and the related link grammars have received a lot of attention in the field of computational linguistics in recent years, since these grammars enable much easier parsing than alternatives that are more complex lexicalized parse structures. There are applications to such popular tasks as machine translation and information retrieval. However, all of the work is concerned with parsing, i.e. inducing a parse structure given a corpus and a grammar, rather than with grammar induction. Some work is concerned with inducing parameters of the grammar from annotated corpora, for example see work by Eisner on dependency parsing [1] or more recent work by McDonald et al. [5] and references therein. It has been pointed out [5] that parsing with dependency grammars is related to Minimal Spanning Tree algorithms in general and in particular Chu-Liu-Edmonds MST algorithm was applied to dependency parsing.

An established computational linguistics textbook has the following to say on the subject [3]: ”… doing grammar induction from scratch is still a difficult, largely unsolved problem, and hence much emphasis has been placed on learning from bracketed corpora.” If grammar is not provided to begin with, parsing has to be done concurrently with learning the grammar. In the presence of grammar, among all the possibilities one needs to pick a syntactic structure consistent with the grammar. In the absence of grammar, it makes sense to appeal to Occam’s razor principle and look for the minimal set of dependencies which are consistent among themselves.

More formally, a dependency grammar consists of a lexicon of terminal symbols (words), and an inventory of dependency relations specifying inter-lexical requirements. A string is generated by a dependency grammar if and only if:

• •

Every word but one (ROOT) is dependent on another word.

• •

No word is dependent on itself either directly or indirectly.

• •

No word is directly dependent on more than one word.

• •

Dependencies do not cross.

Unlike the first three constraints, the last constraint is a linearization constraint, usually introduced to simplify the structure and is empirically problematic. The structure in figure 2.left is an example of so-called projective parse, in which dependency links mapped onto the sentences word sequence do not cross. Figure 2.right illustrates an incorrect parse of the sentence with non-projective dependancies: ”her”$\rightarrow$”suffered” is crossing ”a”$\rightarrow$”predecessor”). While the vast majority of English sentences observe the projectivity constraint, other languages allow much more flexibility in word order. Non-projective structures include wh-relative clauses [7], parentheticals [4], cross-serial constructions of the type found in Dutch and Swiss-German [6], as well as free or relaxed word order languages [8]. Therefore, it is interesting whether grammar induction can be performed without regard to word order.

A truly cross-linguistic formulation of dependency parsing corresponds to finding a spanning tree (parse) in a completely connected subgraph of word nodes and dependency edges. The grammar induction problem in the same setting corresponds to inducing the minimal fully-connected subgraph which contains spanning trees for all sentences in a given corpus. Consider three sentences: ”Her immediate predecessor suffered a nervous breakdown.”, ”Her predecessor suffered a stroke.”, ”It is a nervous breakdown.” Intuitively, the repetition of a word cooccurrence informs us about grammatical co-dependence.

Here is a formulation of the grammar induction problem as an optimization problem: Given a lexicon $V$ and a set of $k$ sentences $S_{1},\ldots,S_{k}$ s.t. $S_{i}\subset V$ (a.k.a. corpus) the objective is to find the most parsimonious combination of dependency structures. i.e. such set of spanning trees for all $S_{i}$ that has the minimal cardinality of a joint set of edges.

In section 2 of this paper, we formally introduce the related graph-theoretic problem. In section 2.1 we show that the problem is hard to approximate within a factor of $c\log n$ for weighted instances, and hard to approximate within some constant factor (APX-hard) for unweighed instances. In section 3, we generalize the problem to matroids. Here we prove that the problem is hard to approximate within a factor of $c\log n$, even for unweighed instances. We conclude with a positive result – an algorithm for the matroid problem which constructs a solution whose cardinality is within $O(\log n)$ of optimal.

2 The Problem for Spanning-Trees

Let $G=(V,E)$ be a graph and let $S_{1},\ldots,S_{k}$ be arbitrary subsets of $V$. Our objective is to find a set of edges $F\subseteq E$ such that

• •

$F$ contains a spanning tree for each induced subgraph $G[S_{i}]$, and

• •

$|F|$ is minimized.

We call this the Min Spanning-Tree Hitting Set problem. Figure 3 illustrates one instance of this problem. A graph $G$ consist of two sub-graphs $G_{1}$ and $G_{2}$. We present one possible correct solution on the left ($|F|$ = 4) and two sample incorrect solutions ($|F|$ = 5) on the right. The Min Spanning-Tree Hitting Set problem may be generalized to include a weight function $w$ on the edges of $G$. The objective for the weighted problem is the same as before, except that we seek to minimize $w(F)$. Notice that the problem initially appears similar to the group Steiner problem [2], since the objective is to connect certain subsets of the nodes. However, our condition on the subgraph is slightly different: we require that the given subsets of nodes are internally connected.

To develop some intuition for this problem, let’s analyze a simple greedy ad-hoc solution: first, assign all the edges weight equivalent to the number of sub-graphs it is included into, i.e. count the frequency of node pairs in the input set; then fragment the graph into subgraphs, keeping the weights and run the standard MST algorithm, to find a spanning tree for each subgraph. Figure 4 presents a counterexample to simple heuristics approaches. The following sub-sets make up the input as indicated via edges of distinct color and pattern in the figure: $\{1,4,5\},\{2,4,5\},$ $\{3,4,5\},\{1,4\},$ $\{1,5\}$, $\{2,4\},\{2,5\},$ $\{3,4\},\{3,5\}$. The optimal solution to this instance does not contain the edge $\{4,5\}$, yet this edge is a member of the most (namely three) sub-sets.

2.1 Hardness for Weighted Instances

We now show that the weighted problem is NP-hard to approximate within a factor of $\log n$. To do so, we exhibit a reduction from Min Hitting Set, which is known to be hard to approximate within $\log n$.

An instance of Min Hitting Set consists of a universe $U=\{u_{1},\ldots,u_{n}\}$ and a collection of sets $\mathcal{T}=\{T_{1},\ldots,T_{m}\}$, each of which is a subset of $U$. We construct a weighted instance of Min Spanning-Tree Hitting Set as follows. Let $r\not\in V$ be a new vertex. We set

[TABLE]

where $K_{U}$ denotes (the edges of) the complete graph on vertex set $U$. The edges belonging to $K_{U}$ have weight $1$ and the edges incident with $r$ have weight $n^{3}$. Let $h$ denote the minimum weight of a Spanning-Tree Hitting Set in $G$. Let $h^{\prime}$ denote the minimum cardinality of a Hitting Set for $\mathcal{T}$.

Claim 1

$h=h^{\prime}\cdot n^{3}+\binom{n}{2}$.

Proof: First we show that h^{\prime}\leq\big{(}h-\binom{n}{2}\big{)}/n^{3} Let $F$ be a spanning-tree hitting set. Clearly $K_{U}\subseteq F$, because of the sets $S_{\{i,j\}}$. So all edges in $F\setminus K_{U}$ are of the form $\{r,u_{i}\}$. Now define $C=\left\{\;u_{i}\,:\,\{r,u_{i}\}\in F\;\right\}$. We now show that $C$ is a hitting set. Consider a set $T_{i}$. Since $F$ contains a spanning tree for $S^{\prime}_{i}$, it must contain some edge $\{r,u_{i}\}$. This shows that $C$ hits the set $T_{i}$.

Now we show that $h\leq h^{\prime}\cdot n^{3}+\binom{n}{2}$. Let $C\subseteq U$ be a hitting set for $\mathcal{T}$. Let F=K_{U}\cup\left\{\;\{r,u_{i}\}\,:\,\mbox{for all u_{i}\in C}\;\right\}. We now show that $F$ is a spanning-tree hitting set. Each set $S_{\{i,j\}}$ is clearly hit by the set $K_{U}$. So consider a set $S^{\prime}_{i}=T_{i}+r$. All edges $\{u_{a},u_{b}\}$ with $a,b\in T_{i}$ are contained in $K_{U}$. Furthermore, since $C$ is a hitting set, there exists an element $u_{a}\in T_{i}\cap C$. This implies that $\{r,u_{a}\}\in F$, and hence $F$ contains a spanning tree for $G[S^{\prime}_{i}]$.

Given an instance $\mathcal{T}$ of Hitting Set, it is NP-hard to decide whether $OPT(\mathcal{T})\leq f(n)$ or $OPT(\mathcal{T})>\alpha\log n\cdot f(n)$ for some constant $\alpha>0$ and some function $f$. To prove $\log n$-hardness of Min Spanning-Tree Hitting Set, we must similarly show that for any instance $y$, there exists a constant $\beta>1$ and a function $g$ such that it is NP-hard to decide whether $OPT(G)\leq g(y)$ or $OPT(G)>\beta\log n\cdot g(y)$.

From our reduction, we know that it is NP-hard to distinguish between

[TABLE]

or

[TABLE]

Now note that

[TABLE]

for some constant $\beta>0$. Letting $g(y)=f(n)\cdot n^{3}+{\textstyle\binom{n}{2}}$, it follows that Min Spanning-Tree Hitting Set is NP-hard to approximate within $\log n$.

2.2 Hardness for Unweighted Instances

We show APX-hardness for the unweighted problem via a reduction from Vertex Cover. The approach is similar to the construction in Section 2.1. Suppose we have an instance $G^{\prime}=(V^{\prime},E^{\prime})$ of the Vertex Cover problem. We use the fact that Vertex Cover is equivalent to Min Hitting Set where $U=E^{\prime}$ and $\mathcal{T}=E^{\prime}$. The construction differs only in that $E^{\prime}$ is used in place of the edge set $K_{U}$; the sets $S_{\{i,j\}}$ are adjusted accordingly. Let $h$ denote the minimum cardinality of a Spanning-Tree Hitting Set in $G$. Let $c$ denote the minimum cardinality of a Vertex Cover in $G^{\prime}$. A claim identical to Claim 1 shows that $h=c+\lvert E^{\prime}\rvert$.

Recall that Vertex Cover is APX-hard even for constant-degree instances; see, e.g., Vazirani [11, §29]. So we may assume that $\lvert E^{\prime}\rvert\leq\frac{d}{2}\lvert V^{\prime}\rvert$. Given an instance $G^{\prime}=(V^{\prime},E^{\prime})$ of Vertex Cover with degree at most some constant $d$, it is NP-hard to decide whether $OPT(G^{\prime})\leq\alpha^{\prime}\lvert V^{\prime}\rvert$ or $OPT(G^{\prime})>\beta^{\prime}\lvert V^{\prime}\rvert$ for some constant $\alpha^{\prime}<\beta^{\prime}$. To prove APX-hardness of Min Spanning-Tree Hitting Set, we must similarly show that for any instance $G$, there exists a constant $\gamma>1$ such that it is NP-hard to decide whether $OPT(G)\leq f(G)$ or $OPT(G)>\gamma f(G)$. From our reduction, we know that it is NP-hard to distinguish between

[TABLE]

or

[TABLE]

Now note that

[TABLE]

which is a constant greater than $1$. Letting $\gamma$ be this constant, and letting $f(y)=\alpha^{\prime}(\lvert V\rvert-1)+(\lvert E\rvert-\lvert V\rvert+1)$, it follows that Min Spanning-Tree Hitting Set is APX-hard.

3 The Problem for Matroids

The Min Spanning-Tree Hitting Set can be rephrased as a question about matroids. Let $E$ be a ground set. Let $M_{i}=(E,\mathcal{I}_{i})$ be a matroid for $1\leq i\leq k$. Our objective is to find $F\subseteq E$ such that

• •

$F$ contains a basis for each $M_{i}$, and

• •

$\lvert F\rvert$ is minimized.

We call this the Minimum Basis Hitting Set problem.

3.1 Connection to Matroid Intersection

Suppose we switch to the dual matroids. Note that $F$ contains a basis for $M_{i}$ if and only $E\setminus F\in\mathcal{I}_{i}^{*}$. Then our objective to find $F^{\prime}\subseteq E$ such that

• •

$F^{\prime}\in\mathcal{I}_{i}^{*}$ for each $i$, and

• •

$\lvert F^{\prime}\rvert$ is maximized.

Suppose that such a set is found, and let $F\,:=\,E\setminus F^{\prime}$. The first property implies that $F$ contains a basis for each $M_{i}$. The second property implies that $\lvert F\rvert$ is minimized. Stated this way, it is precisely the Matroid $k$-Intersection problem. So, from the point of view of exact algorithms, Min Basis Hitting Set and Matroid k-Intersection problems are equivalent. However, this reduction is not approximation-preserving, and implies nothing about approximation algorithms.

3.2 Hardness

Theorem 2

Min Basis-Hitting Set is NP-hard.

Proof:

We do a reduction from the well-known problem Minimum Hitting Set. An instance of this problem consists of a family of sets $\mathcal{C}=\{C_{1},\ldots,C_{k}\}$. The objective is to find a set $F\subseteq E$ such that $F\cap C_{i}\neq\emptyset$ for each $i$. This problem is NP-complete.

Now we reduce it to Minimum Basis Hitting Set. For each set $C_{i}$, set $M_{i}=(E,\mathcal{I}_{i})$ be the matroid where $\mathcal{I}_{i}=\left\{\;\{c\}\,:\,c\in C_{i}\;\right\}\cup\{\emptyset\}$. That is, $M_{i}$ is the rank-1 uniform matroid on $C_{i}$. So a basis hitting set for these matroids corresponds precisely to a hitting set for the the sets $\mathcal{C}$.

Corollary 3

Min Basis Hitting Set is NP-hard to approximate with $c\log n$ for some positive constant $c$.

Proof: It is well-known that Min Hitting Set is equivalent to Set Cover, and is therefore NP-hard to approximate within $c\log n$ for some positive constant $c$. Since reduction given in Theorem 2 is approximation preserving, the same hardness applies to Min Basis Hitting Set.

3.3 An Approximation Algorithm

We consider the greedy algorithm for the Min Basis Hitting Set problem. Let $\mathcal{O}\subseteq E$ denote an optimum solution. Let $\operatorname{rank}_{j}$ denote the rank function for matroid $M_{j}$ and let $r_{j}$ be the rank of $M_{j}$, i.e., $r_{j}=\operatorname{rank}_{j}(E)$. Let $F_{i}$ denote the set that has been chosen after the $i{{}^{\textrm{th}}}$ step of the algorithm. Initially, we have $F_{0}=\emptyset$. For $S\subseteq E$, let P(S,e)=\sum_{j=1}^{k}\big{(}\operatorname{rank}_{j}(S+e)-\operatorname{rank}_{j}(S)\big{)}; intuitively, this is the total “profit” obtained, or rank that is hit, by adding $e$ to $S$. Let $R_{i}$ denote \sum_{j=1}^{k}\big{(}r_{j}-\operatorname{rank}_{j}(F_{i})\big{)}; intuitively, if the algorithm has chosen a set $F_{i}$, then $R_{i}$ is the total amount of “residual rank” that remains to be hit.

Consider the $i{{}^{\textrm{th}}}$ step of the algorithm. Let’s denote the profit obtained by choosing $e_{i}$ by $p_{i}=\max_{e\not\in F_{i-1}}P(F_{i-1},e)$. The greedy algorithm chooses an element $e_{i}\not\in F_{i-1}$ achieving the maximum profit. We now analyze the efficiency of this algorithm. Let $\mathcal{O}_{i}$ be a minimum-cardinality set that contains $F_{i}$ and is a basis hitting set.

For any set $S\supseteq F_{i}$ and any $e\not\in S$, we have (by submodularity):

[TABLE]

This implies that each edge in $\mathcal{O}_{i}\setminus F_{i}$ has profit at most $p_{i}$. Since $\mathcal{O}_{i}$ must ultimately hit all of the residual rank, but each element hits at most $p_{i}$, we have $R_{i-1}\leq p_{i}\cdot\lvert\mathcal{O}_{i}\setminus F_{i}\rvert$.

Now, note that $\lvert\mathcal{O}_{i}\setminus F_{i}\rvert\leq\lvert\mathcal{O}\rvert$. This is is because of the non-decreasing property of $\operatorname{rank}_{j}$: if $\mathcal{O}$ is a basis hitting set then so is $\mathcal{O}\cup F_{i}$. This observation yields the inequality $1\leq p_{i}\cdot\lvert\mathcal{O}\rvert/R_{i-1}$. Suppose that the greedy algorithm halts with a solution of cardinality $s$. Then we have

[TABLE]

Here, the last inequality follows from the fact that $R_{i}=R_{i-1}-p_{i}$ for $1\leq i\leq s$. Note that $R_{0}=\sum_{j=1}^{k}r_{j}$ is the total rank of the given matroids.

The preceding argument shows that the greedy algorithm has approximation ratio $O(\log n)$, where $n$ is the length of the input. Table 1 presents description of the algorithm. Informally speaking, the algorithm could be explain as follows: Estimate potential number of sub-graphs each edge would contribute to if used. Loop through all edges, adding in (greedily) the edge which contributes to the most spanning trees, then re-calculate potential contributions.

3.4 Contrast with Matroid Union

Consider the matroid union problem for matroids $M_{i}^{*}$. The matroid union problem is:

[TABLE]

But note that $S_{i}\in\mathcal{I}_{i}^{*}$ iff $r_{i}(V\setminus S_{i})=r_{i}(V)$. In other words, $S_{i}\in\mathcal{I}_{i}^{*}$ iff $\bar{S_{i}}$ contains a basis for $M_{i}$. And maximizing the size of the union is the same as minimizing the size of the complement of the union. So an equivalent problem is:

[TABLE]

The minimum does not change if we assume that $\overline{S_{i}}$ in fact is a basis. So, letting $T_{i}$ denote $\bar{S_{i}}$, we obtain the equivalent problem:

[TABLE]

This problem is solvable in polynomial time, because it is just matroid union in disguise. It is quite similar to the Minimum Basis Hitting Set problem, except that it has an “intersection” rather than an “union”.

3.5 Empirical study

We ran preliminary experiments with the approximation algorithm on adult child-directed speech from the CHILDES corpus [9]. These experiments demonstrated that the algorithm performs better than the baseline adjacency heuristic because of its ability to pick out non-adjacent dependencies. For example, the sentence ”Is that a woof?” is parsed into the following set of links: woof-is, that-is, a-woof. The links correspond to the correct parse tree of the sentence, In contrast, the baseline adjacency heuristic would parse the sentence into is-that; that-a; and a-woof, which fails to capture the dependence between the predicate noun ”woof” and the verb, and postulates a non-existent dependency between the determiner ”a” and the subject ”that”. However, more work is needed to thoroughly assess the performance. In particular, one problem for direct application is the presence of repeated words in the sentence. The current implementation avoids the issue of repeated words in its entirety, by filtering the input text. An alternative approach is to erase the edges among repeated words from the original fully connected graph. This assumes that no word can be a dependent of itself, which might be a problem in some contexts (e.g. ”I know that you know”). Related work which was not completed at the time of writing this manuscript seeks to incorporate adjacency as a soft linguistic constraint on the graph by increasing initial weight edges of adjacent words.

4 Discussion

We presented some theoretical results for a problem on graphs which is inspired by the unsupervised link grammar induction problem from linguistics. Numerous possible directions for the future work would include searching for more efficient approximation algorithms under various additional constraints on admissible spanning trees, as well as characterizing instances of the problem which could be solved efficiently. Another possible direction is allowing ”ungrammatical” corpus as input, e.g. searching efficiently for partial solutions, where several sentences remain unparsed or not fully parsed. Another direction is to look for a solution to a directed graph analog of the problem considered here, which would require finding minimal set of arborescences and relate to the directed dependency parsing. One other question which remains open is an edge weighing scheme which would reflect syntactic consideration and particular language-related constraints, as in the so-called Optimality Theory [10].

Exploring relation of this problem to other application would be interesting. One such example could be an autonomous network design, where an objective is to efficiently design a network that must connect joint units of organizations which do not necessarily trust each other and want to maintain their own skeletal sub-network in case their partner’s links fail.