A Structured Language Model

TL;DR

This paper introduces a structured language model that incorporates syntactic parsing to improve the understanding of long-distance dependencies and enhances predictive accuracy in language modeling.

Contribution

It presents a novel probabilistic language model that integrates syntactic structure and headword annotations for better long-range dependency modeling.

Findings

Demonstrates improved prediction of long-distance dependencies

Provides a probabilistic framework for joint word and structure sequences

Evaluates the model's predictive power through experiments

Abstract

The paper presents a language model that develops syntactic structure and uses it to extract meaningful information from the word history, thus enabling the use of long distance dependencies. The model assigns probability to every joint sequence of words - binary-parse-structure with headword annotation. The model, its probabilistic parametrization, and a set of experiments meant to evaluate its predictive power are presented.