Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars

TL;DR

This paper introduces an unsupervised method for discontinuous constituency parsing using mildly context-sensitive grammars, leveraging LCFRS formalism, tensor decomposition, and embedding-based parameterization to induce linguistically meaningful trees.

Contribution

It proposes a novel unsupervised parsing approach with fixed rule structures, scalable nonterminal modeling, and reduced computational complexity for discontinuous structures.

Findings

Successfully induced linguistically meaningful trees for German and Dutch.

Demonstrated the effectiveness of tensor decomposition in scaling nonterminal numbers.

Reduced parsing complexity from O(n^6) to O(n^5).

Abstract

We study grammar induction with mildly context-sensitive grammars for unsupervised discontinuous parsing. Using the probabilistic linear context-free rewriting system (LCFRS) formalism, our approach fixes the rule structure in advance and focuses on parameter learning with maximum likelihood. To reduce the computational complexity of both parsing and parameter estimation, we restrict the grammar formalism to LCFRS-2 (i.e., binary LCFRS with fan-out two) and further discard rules that require O(n^6) time to parse, reducing inference to O(n^5). We find that using a large number of nonterminals is beneficial and thus make use of tensor decomposition-based rank-space dynamic programming with an embedding-based parameterization of rule probabilities to scale up the number of nonterminals. Experiments on German and Dutch show that our approach is able to induce linguistically meaningful trees with continuous and discontinuous structures