Dynamic Integration of Background Knowledge in Neural NLU Systems

TL;DR

This paper presents a novel neural NLU architecture that dynamically incorporates explicit background knowledge from free-text sources, improving understanding in tasks like DQA and RTE.

Contribution

It introduces a general-purpose reading module that dynamically integrates background knowledge into neural NLU models, enhancing their flexibility and semantic understanding.

Findings

Improved performance on document question answering and RTE tasks.

Model effectively exploits background knowledge in a semantically appropriate manner.

Demonstrates flexibility and effectiveness of dynamic knowledge integration.

Abstract

Common-sense and background knowledge is required to understand natural language, but in most neural natural language understanding (NLU) systems, this knowledge must be acquired from training corpora during learning, and then it is static at test time. We introduce a new architecture for the dynamic integration of explicit background knowledge in NLU models. A general-purpose reading module reads background knowledge in the form of free-text statements (together with task-specific text inputs) and yields refined word representations to a task-specific NLU architecture that reprocesses the task inputs with these representations. Experiments on document question answering (DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness and flexibility of the approach. Analysis shows that our model learns to exploit knowledge in a semantically appropriate way.