Ask Me Even More: Dynamic Memory Tensor Networks (Extended Model)

TL;DR

This paper introduces Dynamic Memory Tensor Networks, an extension of Memory Networks with improved attention mechanisms, achieving significant performance gains in question answering tasks with limited and weakly supervised data.

Contribution

The paper proposes a novel extension to Dynamic Memory Networks called DMTN, enhancing attention mechanisms for better performance in low-data QA scenarios.

Findings

Over 80% improvement in task success rate over baseline DMN

20% more tasks passed compared to state-of-the-art End-to-End Memory Network

Effective in weakly supervised, low-data environments

Abstract

We examine Memory Networks for the task of question answering (QA), under common real world scenario where training examples are scarce and under weakly supervised scenario, that is only extrinsic labels are available for training. We propose extensions for the Dynamic Memory Network (DMN), specifically within the attention mechanism, we call the resulting Neural Architecture as Dynamic Memory Tensor Network (DMTN). Ultimately, we see that our proposed extensions results in over 80% improvement in the number of task passed against the baselined standard DMN and 20% more task passed compared to state-of-the-art End-to-End Memory Network for Facebook's single task weakly trained 1K bAbi dataset.