Portability of Syntactic Structure for Language Modeling

TL;DR

This study explores how well statistical syntactic knowledge can be transferred across domains in language modeling, showing that porting from a different domain can outperform in-domain data in reducing word error rates.

Contribution

It demonstrates that porting syntactic statistics from one domain to another can outperform traditional in-domain parsing methods in language modeling.

Findings

Ported SLM reduces WER by 0.4% absolute and 7% relative.

Ported SLM outperforms rule-based parser and manually parsed data.

Porting improves language model performance despite modest perplexity gains.

Abstract

The paper presents a study on the portability of statistical syntactic knowledge in the framework of the structured language model (SLM). We investigate the impact of porting SLM statistics from the Wall Street Journal (WSJ) to the Air Travel Information System (ATIS) domain. We compare this approach to applying the Microsoft rule-based parser (NLPwin) for the ATIS data and to using a small amount of data manually parsed at UPenn for gathering the intial SLM statistics. Surprisingly, despite the fact that it performs modestly in perplexity (PPL), the model initialized on WSJ parses outperforms the other initialization methods based on in-domain annotated data, achieving a significant 0.4% absolute and 7% relative reduction in word error rate (WER) over a baseline system whose word error rate is 5.8%; the improvement measured relative to the minimum WER achievable on the N-best lists we worked with is 12%.