Expoiting Syntactic Structure for Language Modeling

TL;DR

This paper introduces a language model that learns syntactic structures to better capture long-distance dependencies, improving prediction accuracy over traditional trigram models and suitable for speech recognition applications.

Contribution

It presents a novel syntactic structure-based language model that integrates parse structures into probability estimation, enhancing long-range dependency modeling.

Findings

Improved prediction accuracy over trigram models

Effective use of syntactic structures for language modeling

Model suitable for speech recognition tasks

Abstract

The paper presents a language model that develops syntactic structure and uses it to extract meaningful information from the word history, thus enabling the use of long distance dependencies. The model assigns probability to every joint sequence of words--binary-parse-structure with headword annotation and operates in a left-to-right manner --- therefore usable for automatic speech recognition. The model, its probabilistic parameterization, and a set of experiments meant to evaluate its predictive power are presented; an improvement over standard trigram modeling is achieved.