Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling

TL;DR

This paper introduces a sequence labeling framework that uses sparse indicator features from dense word embeddings, achieving near state-of-the-art results in multilingual POS tagging and NER with minimal training data.

Contribution

It presents a novel approach that leverages sparse coding of neural embeddings for effective multilingual sequence labeling without modifying the original embeddings.

Findings

Achieves near state-of-the-art performance in POS tagging and NER across multiple languages.

Maintains over 89.8% of its accuracy with only 1.2% of training data.

Uses only a few thousand sparse features without altering word representations.

Abstract

In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-of-the art performance for both part-of-speech tagging and named entity recognition for a variety of languages. Our model relies only on a few thousand sparse coding-derived features, without applying any modification of the word representations employed for the different tasks. The proposed model has favorable generalization properties as it retains over 89.8% of its average POS tagging accuracy when trained at 1.2% of the total available training data, i.e.~150 sentences per language.