One Model to Rule them all: Multitask and Multilingual Modelling for Lexical Analysis

TL;DR

This paper explores a unified model for multitask and multilingual lexical analysis in NLP, aiming to improve performance on low-resource languages by leveraging shared knowledge across tasks and languages.

Contribution

It introduces a novel approach that combines multitask and multilingual learning for lexical analysis, enhancing NLP capabilities for low-resource languages.

Findings

Shared representations improve low-resource language performance

Multitask learning benefits multiple lexical tasks simultaneously

Multilingual models outperform monolingual counterparts on several benchmarks

Abstract

When learning a new skill, you take advantage of your preexisting skills and knowledge. For instance, if you are a skilled violinist, you will likely have an easier time learning to play cello. Similarly, when learning a new language you take advantage of the languages you already speak. For instance, if your native language is Norwegian and you decide to learn Dutch, the lexical overlap between these two languages will likely benefit your rate of language acquisition. This thesis deals with the intersection of learning multiple tasks and learning multiple languages in the context of Natural Language Processing (NLP), which can be defined as the study of computational processing of human language. Although these two types of learning may seem different on the surface, we will see that they share many similarities. The traditional approach in NLP is to consider a single task for a single language at a time. However, recent advances allow for broadening this approach, by considering data for multiple tasks and languages simultaneously. This is an important approach to explore further as the key to improving the reliability of NLP, especially for low-resource languages, is to take advantage of all relevant data whenever possible. In doing so, the hope is that in the long term, low-resource languages can benefit from the advances made in NLP which are currently to a large extent reserved for high-resource languages. This, in turn, may then have positive consequences for, e.g., language preservation, as speakers of minority languages will have a lower degree of pressure to using high-resource languages. In the short term, answering the specific research questions posed should be of use to NLP researchers working towards the same goal.