Multilingual Controllable Transformer-Based Lexical Simplification

TL;DR

This paper introduces mTLS, a multilingual, controllable Transformer-based lexical simplification system that leverages language-specific prompts and pre-trained language models to generate simpler word alternatives, outperforming existing models across multiple languages.

Contribution

The work presents a novel multilingual LS model using language-specific prefixes and control tokens with T5, achieving state-of-the-art results and demonstrating effectiveness across English, Spanish, and Portuguese.

Findings

Outperforms previous state-of-the-art models like LSBert and ConLS

Achieves competitive results on TSAR-2022 multilingual LS dataset

Outperforms GPT-3 on several metrics in English LS

Abstract

Text is by far the most ubiquitous source of knowledge and information and should be made easily accessible to as many people as possible; however, texts often contain complex words that hinder reading comprehension and accessibility. Therefore, suggesting simpler alternatives for complex words without compromising meaning would help convey the information to a broader audience. This paper proposes mTLS, a multilingual controllable Transformer-based Lexical Simplification (LS) system fined-tuned with the T5 model. The novelty of this work lies in the use of language-specific prefixes, control tokens, and candidates extracted from pre-trained masked language models to learn simpler alternatives for complex words. The evaluation results on three well-known LS datasets -- LexMTurk, BenchLS, and NNSEval -- show that our model outperforms the previous state-of-the-art models like LSBert and ConLS. Moreover, further evaluation of our approach on the part of the recent TSAR-2022 multilingual LS shared-task dataset shows that our model performs competitively when compared with the participating systems for English LS and even outperforms the GPT-3 model on several metrics. Moreover, our model obtains performance gains also for Spanish and Portuguese.