Effective Character-augmented Word Embedding for Machine Reading Comprehension

TL;DR

This paper introduces a character-augmented word embedding method that enhances machine reading comprehension by effectively integrating character-level information, especially benefiting rare words, leading to improved performance on benchmarks.

Contribution

It proposes a novel character-augmented reader that attends to character-level representations, significantly improving upon existing models in machine reading comprehension.

Findings

Significant performance improvement over baselines

Effective handling of rare words

Outperforms state-of-the-art on benchmarks

Abstract

Machine reading comprehension is a task to model relationship between passage and query. In terms of deep learning framework, most of state-of-the-art models simply concatenate word and character level representations, which has been shown suboptimal for the concerned task. In this paper, we empirically explore different integration strategies of word and character embeddings and propose a character-augmented reader which attends character-level representation to augment word embedding with a short list to improve word representations, especially for rare words. Experimental results show that the proposed approach helps the baseline model significantly outperform state-of-the-art baselines on various public benchmarks.