Words or Characters? Fine-grained Gating for Reading Comprehension
Zhilin Yang, Bhuwan Dhingra, Ye Yuan, Junjie Hu, William W. Cohen,, Ruslan Salakhutdinov

TL;DR
This paper introduces a dynamic fine-grained gating mechanism to better combine word and character representations, significantly improving reading comprehension performance and generalizing to other NLP tasks.
Contribution
The paper proposes a novel fine-grained gating method for integrating word and character features, enhancing reading comprehension models and demonstrating broad applicability.
Findings
Achieves state-of-the-art results on the Children's Book Test dataset.
Improves social media tag prediction performance.
Outperforms previous methods in combining word and character representations.
Abstract
Previous work combines word-level and character-level representations using concatenation or scalar weighting, which is suboptimal for high-level tasks like reading comprehension. We present a fine-grained gating mechanism to dynamically combine word-level and character-level representations based on properties of the words. We also extend the idea of fine-grained gating to modeling the interaction between questions and paragraphs for reading comprehension. Experiments show that our approach can improve the performance on reading comprehension tasks, achieving new state-of-the-art results on the Children's Book Test dataset. To demonstrate the generality of our gating mechanism, we also show improved results on a social media tag prediction task.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Text Readability and Simplification
