How to Evaluate Word Representations of Informal Domain?

TL;DR

This paper introduces a new method for evaluating word embeddings in informal domains like Twitter by automatically extracting variant spelling pairs from UrbanDictionary, reducing the need for text normalization.

Contribution

It presents an automatic approach to derive variant spelling pairs for informal language, enabling more direct evaluation of word representations without normalization.

Findings

Large list of variant spelling pairs extracted from UrbanDictionary

Facilitates direct evaluation of non-standard word embeddings

Reduces reliance on traditional text normalization pipelines

Abstract

Diverse word representations have surged in most state-of-the-art natural language processing (NLP) applications. Nevertheless, how to efficiently evaluate such word embeddings in the informal domain such as Twitter or forums, remains an ongoing challenge due to the lack of sufficient evaluation dataset. We derived a large list of variant spelling pairs from UrbanDictionary with the automatic approaches of weakly-supervised pattern-based bootstrapping and self-training linear-chain conditional random field (CRF). With these extracted relation pairs we promote the odds of eliding the text normalization procedure of traditional NLP pipelines and directly adopting representations of non-standard words in the informal domain. Our code is available.