Controllable Lexical Simplification for English
Kim Cheng Sheang, Daniel Ferr\'es, Horacio Saggion

TL;DR
This paper introduces ConLS, a controllable lexical simplification system based on T5, which achieves comparable or better performance than current state-of-the-art models across multiple datasets, with insights into control token effectiveness.
Contribution
The paper presents the first application of Transformer fine-tuning for lexical simplification, introducing controllability and demonstrating competitive results.
Findings
ConLS performs comparably to LSBert on three datasets.
Control tokens significantly influence model outputs.
ConLS outperforms LSBert in some evaluation cases.
Abstract
Fine-tuning Transformer-based approaches have recently shown exciting results on sentence simplification task. However, so far, no research has applied similar approaches to the Lexical Simplification (LS) task. In this paper, we present ConLS, a Controllable Lexical Simplification system fine-tuned with T5 (a Transformer-based model pre-trained with a BERT-style approach and several other tasks). The evaluation results on three datasets (LexMTurk, BenchLS, and NNSeval) have shown that our model performs comparable to LSBert (the current state-of-the-art) and even outperforms it in some cases. We also conducted a detailed comparison on the effectiveness of control tokens to give a clear view of how each token contributes to the model.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsText Readability and Simplification · Natural Language Processing Techniques · Topic Modeling
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Inverse Square Root Schedule · Dropout · Dense Connections · Attention Dropout · Linear Layer · Layer Normalization · Multi-Head Attention · Gated Linear Unit
