Modular Approach to Machine Reading Comprehension: Mixture of Task-Aware Experts

TL;DR

This paper introduces a Mixture of Task-Aware Experts Network for machine reading comprehension that enhances common-sense understanding and reduces overfitting by training specialized expert networks on focused tasks.

Contribution

It proposes a novel mixture-of-experts framework that captures diverse relationships and enforces task-specific learning to improve comprehension performance.

Findings

Achieved state-of-the-art results on a small dataset.

Effectively captures different relationship types in passages.

Reduces overfitting through task-aware training.

Abstract

In this work we present a Mixture of Task-Aware Experts Network for Machine Reading Comprehension on a relatively small dataset. We particularly focus on the issue of common-sense learning, enforcing the common ground knowledge by specifically training different expert networks to capture different kinds of relationships between each passage, question and choice triplet. Moreover, we take inspi ration on the recent advancements of multitask and transfer learning by training each network a relevant focused task. By making the mixture-of-networks aware of a specific goal by enforcing a task and a relationship, we achieve state-of-the-art results and reduce over-fitting.