Making Neural Machine Reading Comprehension Faster

TL;DR

This paper presents a method to accelerate neural machine reading comprehension by applying knowledge distillation to create smaller, faster models based on BERT, aiming to improve inference speed without sacrificing accuracy.

Contribution

The study introduces a knowledge distillation approach to develop faster, smaller BERT-based models for machine reading comprehension, enhancing inference efficiency.

Findings

Smaller models achieved faster inference times.

Distilled models maintained comparable accuracy to larger models.

Compared favorably with existing speed-optimized models.

Abstract

This study aims at solving the Machine Reading Comprehension problem where questions have to be answered given a context passage. The challenge is to develop a computationally faster model which will have improved inference time. State of the art in many natural language understanding tasks, BERT model, has been used and knowledge distillation method has been applied to train two smaller models. The developed models are compared with other models which have been developed with the same intention.