Automatically Selecting Useful Phrases for Dialogue Act Tagging

TL;DR

This paper introduces a novel method for automatically selecting useful phrases for dialogue act tagging, outperforming traditional and manual cue phrase selection techniques.

Contribution

A new deviation-based phrase selection method combined with lexical filtering that improves dialogue act tagging accuracy.

Findings

Outperforms manual cue phrase selection

Better than exhaustive phrase sets

Superior to mutual information and information gain methods

Abstract

We present an empirical investigation of various ways to automatically identify phrases in a tagged corpus that are useful for dialogue act tagging. We found that a new method (which measures a phrase's deviation from an optimally-predictive phrase), enhanced with a lexical filtering mechanism, produces significantly better cues than manually-selected cue phrases, the exhaustive set of phrases in a training corpus, and phrases chosen by traditional metrics, like mutual information and information gain.