FonMTL: Towards Multitask Learning for the Fon Language

TL;DR

This paper introduces FonMTL, a multitask learning approach for the low-resource Fon language, improving NER and POS tagging performance by sharing representations across tasks and demonstrating competitive results against multilingual models.

Contribution

First exploration of multitask learning for Fon language NLP, leveraging shared representations for NER and POS tasks, with insights on loss weighting strategies.

Findings

Multitask learning improves Fon NER and POS performance.

Equal loss weighting strategy yields best results.

Model outperforms some multilingual pretrained models.

Abstract

The Fon language, spoken by an average 2 million of people, is a truly low-resourced African language, with a limited online presence, and existing datasets (just to name but a few). Multitask learning is a learning paradigm that aims to improve the generalization capacity of a model by sharing knowledge across different but related tasks: this could be prevalent in very data-scarce scenarios. In this paper, we present the first explorative approach to multitask learning, for model capabilities enhancement in Natural Language Processing for the Fon language. Specifically, we explore the tasks of Named Entity Recognition (NER) and Part of Speech Tagging (POS) for Fon. We leverage two language model heads as encoders to build shared representations for the inputs, and we use linear layers blocks for classification relative to each task. Our results on the NER and POS tasks for Fon, show competitive (or better) performances compared to several multilingual pretrained language models finetuned on single tasks. Additionally, we perform a few ablation studies to leverage the efficiency of two different loss combination strategies and find out that the equal loss weighting approach works best in our case. Our code is open-sourced at https://github.com/bonaventuredossou/multitask_fon.