Leveraging Knowledge Bases in LSTMs for Improving Machine Reading

TL;DR

This paper introduces KBLSTM, a neural model that uses continuous knowledge base representations with attention mechanisms to improve machine reading tasks like entity and event extraction, surpassing previous state-of-the-art results.

Contribution

The paper presents KBLSTM, a novel neural network that effectively integrates continuous KB representations with text using attention, reducing reliance on task-specific feature engineering.

Findings

Achieved state-of-the-art accuracy on ACE2005 for entity extraction

Improved event extraction performance over previous methods

Demonstrated effective knowledge integration with attention mechanisms

Abstract

This paper focuses on how to take advantage of external knowledge bases (KBs) to improve recurrent neural networks for machine reading. Traditional methods that exploit knowledge from KBs encode knowledge as discrete indicator features. Not only do these features generalize poorly, but they require task-specific feature engineering to achieve good performance. We propose KBLSTM, a novel neural model that leverages continuous representations of KBs to enhance the learning of recurrent neural networks for machine reading. To effectively integrate background knowledge with information from the currently processed text, our model employs an attention mechanism with a sentinel to adaptively decide whether to attend to background knowledge and which information from KBs is useful. Experimental results show that our model achieves accuracies that surpass the previous state-of-the-art results for both entity extraction and event extraction on the widely used ACE2005 dataset.