Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension

TL;DR

This paper introduces auxiliary tasks during fine-tuning of multilingual models to improve answer boundary detection in cross-lingual machine reading comprehension, leveraging cross-lingual data and knowledge masking.

Contribution

It proposes two novel auxiliary tasks for fine-tuning that enhance answer boundary detection in multilingual MRC models, especially for low-resource languages.

Findings

Significant improvement in cross-lingual MRC performance

Effective use of phrase boundary supervision techniques

Enhanced transferability across languages

Abstract

Multilingual pre-trained models could leverage the training data from a rich source language (such as English) to improve performance on low resource languages. However, the transfer quality for multilingual Machine Reading Comprehension (MRC) is significantly worse than sentence classification tasks mainly due to the requirement of MRC to detect the word level answer boundary. In this paper, we propose two auxiliary tasks in the fine-tuning stage to create additional phrase boundary supervision: (1) A mixed MRC task, which translates the question or passage to other languages and builds cross-lingual question-passage pairs; (2) A language-agnostic knowledge masking task by leveraging knowledge phrases mined from web. Besides, extensive experiments on two cross-lingual MRC datasets show the effectiveness of our proposed approach.