Optimizing Spectral Learning for Parsing

TL;DR

This paper introduces a global optimization approach for spectral learning of latent-variable PCFGs, demonstrating improved parsing accuracy across multiple languages by considering interactions between nonterminals.

Contribution

It presents a novel search algorithm for optimizing latent states globally in spectral methods, challenging the belief that states can be set independently.

Findings

Global optimization improves parsing results

Spectral methods perform comparably to EM techniques

Effective across diverse morphologically rich languages

Abstract

We describe a search algorithm for optimizing the number of latent states when estimating latent-variable PCFGs with spectral methods. Our results show that contrary to the common belief that the number of latent states for each nonterminal in an L-PCFG can be decided in isolation with spectral methods, parsing results significantly improve if the number of latent states for each nonterminal is globally optimized, while taking into account interactions between the different nonterminals. In addition, we contribute an empirical analysis of spectral algorithms on eight morphologically rich languages: Basque, French, German, Hebrew, Hungarian, Korean, Polish and Swedish. Our results show that our estimation consistently performs better or close to coarse-to-fine expectation-maximization techniques for these languages.