Automatic Error Detection in Part of Speech Tagging

TL;DR

This paper presents a technique for detecting errors in part of speech tagging using Hidden Markov Models by comparing observable values with a threshold, improving accuracy at the cost of efficiency.

Contribution

It introduces a novel error detection method for HMM taggers based on threshold comparison, enhancing tagging accuracy.

Findings

Technique effectively detects tagging errors

Empirical results validate the approach

Guidelines for threshold selection provided

Abstract

A technique for detecting errors made by Hidden Markov Model taggers is described, based on comparing observable values of the tagging process with a threshold. The resulting approach allows the accuracy of the tagger to be improved by accepting a lower efficiency, defined as the proportion of words which are tagged. Empirical observations are presented which demonstrate the validity of the technique and suggest how to choose an appropriate threshold.