Charagram: Embedding Words and Sentences via Character n-grams

TL;DR

Charagram introduces a simple character n-gram based embedding method for words and sentences, outperforming complex neural models on similarity tasks with state-of-the-art results.

Contribution

The paper proposes a straightforward character n-gram approach for embedding textual sequences that surpasses complex neural architectures in similarity evaluations.

Findings

Outperforms character-level RNN and CNN models on similarity tasks

Achieves new state-of-the-art performance on several benchmarks

Demonstrates effectiveness of simple character n-gram embeddings

Abstract

We present Charagram embeddings, a simple approach for learning character-based compositional models to embed textual sequences. A word or sentence is represented using a character n-gram count vector, followed by a single nonlinear transformation to yield a low-dimensional embedding. We use three tasks for evaluation: word similarity, sentence similarity, and part-of-speech tagging. We demonstrate that Charagram embeddings outperform more complex architectures based on character-level recurrent and convolutional neural networks, achieving new state-of-the-art performance on several similarity tasks.