Weak Semi-Markov CRFs for NP Chunking in Informal Text

TL;DR

This paper presents a new annotated SMS corpus for noun phrase chunking and introduces a semi-Markov CRF variant that achieves comparable accuracy with faster processing.

Contribution

The paper develops a new annotated corpus for informal text and proposes a novel semi-CRF variant that improves efficiency for NP chunking.

Findings

The new corpus contains 76,490 noun phrases from SMS messages.

The semi-CRF variant runs significantly faster than traditional models.

Accuracy of the new model is comparable to existing semi-CRF approaches.

Abstract

This paper introduces a new annotated corpus based on an existing informal text corpus: the NUS SMS Corpus (Chen and Kan, 2013). The new corpus includes 76,490 noun phrases from 26,500 SMS messages, annotated by university students. We then explored several graphical models, including a novel variant of the semi-Markov conditional random fields (semi-CRF) for the task of noun phrase chunking. We demonstrated through empirical evaluations on the new dataset that the new variant yielded similar accuracy but ran in significantly lower running time compared to the conventional semi-CRF.