A random forest system combination approach for error detection in digital dictionaries

TL;DR

This paper presents a hybrid approach using random forests to automate error detection in digitized bilingual dictionaries, effectively combining rule-based, feature-based, and language model methods with minimal training data.

Contribution

It introduces a novel application of random forests to combine unsupervised error detection methods in digital dictionaries, reducing the need for extensive training data.

Findings

Random forests effectively combine multiple error detection methods.

Unsupervised methods perform comparably to supervised ones in this context.

A small amount of training data suffices for effective error detection.

Abstract

When digitizing a print bilingual dictionary, whether via optical character recognition or manual entry, it is inevitable that errors are introduced into the electronic version that is created. We investigate automating the process of detecting errors in an XML representation of a digitized print dictionary using a hybrid approach that combines rule-based, feature-based, and language model-based methods. We investigate combining methods and show that using random forests is a promising approach. We find that in isolation, unsupervised methods rival the performance of supervised methods. Random forests typically require training data so we investigate how we can apply random forests to combine individual base methods that are themselves unsupervised without requiring large amounts of training data. Experiments reveal empirically that a relatively small amount of data is sufficient and can potentially be further reduced through specific selection criteria.