Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

TL;DR

This paper explores the use of cascading language and task adapters in multilingual transformer models to enhance question answering performance in low-resource languages, demonstrating significant improvements.

Contribution

It introduces a novel approach of stacking language and task adapters for low-resource question answering, including a zero-shot transfer learning method.

Findings

Stacked adapters significantly improve low-resource language QA performance.

Zero-shot transfer learning with adapters is effective for low-resource languages.

Multilingual transformer models benefit from adapter combinations across multiple languages.

Abstract

Transformer based architectures have shown notable results on many down streaming tasks including question answering. The availability of data, on the other hand, impedes obtaining legitimate performance for low-resource languages. In this paper, we investigate the applicability of pre-trained multilingual models to improve the performance of question answering in low-resource languages. We tested four combinations of language and task adapters using multilingual transformer architectures on seven languages similar to MLQA dataset. Additionally, we have also proposed zero-shot transfer learning of low-resource question answering using language and task adapters. We observed that stacking the language and the task adapters improves the multilingual transformer models' performance significantly for low-resource languages.