Explain to me like I am five -- Sentence Simplification Using Transformers
Aman Agarwal

TL;DR
This paper presents a sentence simplification method using pre-trained transformer models, specifically GPT-2 and BERT, achieving state-of-the-art results without relying on external linguistic resources.
Contribution
The study demonstrates that pure transformer-based models can effectively simplify sentences, surpassing previous methods that used external linguistic databases or control tokens.
Findings
Achieved a SARI score of 46.80 on the Mechanical Turk dataset.
Outperformed previous state-of-the-art results in sentence simplification.
Validated the effectiveness of using only pre-trained transformers for the task.
Abstract
Sentence simplification aims at making the structure of text easier to read and understand while maintaining its original meaning. This can be helpful for people with disabilities, new language learners, or those with low literacy. Simplification often involves removing difficult words and rephrasing the sentence. Previous research have focused on tackling this task by either using external linguistic databases for simplification or by using control tokens for desired fine-tuning of sentences. However, in this paper we purely use pre-trained transformer models. We experiment with a combination of GPT-2 and BERT models, achieving the best SARI score of 46.80 on the Mechanical Turk dataset, which is significantly better than previous state-of-the-art results. The code can be found at https://github.com/amanbasu/sentence-simplification.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsText Readability and Simplification · Topic Modeling · Natural Language Processing Techniques
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Multi-Head Attention · Attention Is All You Need · Cosine Annealing · Byte Pair Encoding · Linear Layer · Linear Warmup With Cosine Annealing · Discriminative Fine-Tuning · Adam · Softmax
