On Unsupervised Training of Link Grammar Based Language Models

TL;DR

This paper investigates the requirements for unsupervised training of link grammar-based language models, introducing formalism and analyzing the limitations of bigram approaches in capturing linguistic context.

Contribution

It introduces a termination tags formalism for link grammar models and highlights the importance of context in unsupervised language learning, challenging previous bigram-based methods.

Findings

Proposes a new statistical link grammar formalism for language generation.

Shows that bigram-based approaches ignore contextual properties of language.

Highlights flaws in previous unsupervised learning methods relying solely on lexical attraction.

Abstract

In this short note we explore what is needed for the unsupervised training of graph language models based on link grammars. First, we introduce the ter-mination tags formalism required to build a language model based on a link grammar formalism of Sleator and Temperley [21] and discuss the influence of context on the unsupervised learning of link grammars. Second, we pro-pose a statistical link grammar formalism, allowing for statistical language generation. Third, based on the above formalism, we show that the classical dissertation of Yuret [25] on discovery of linguistic relations using lexical at-traction ignores contextual properties of the language, and thus the approach to unsupervised language learning relying just on bigrams is flawed. This correlates well with the unimpressive results in unsupervised training of graph language models based on bigram approach of Yuret.