Explain to me like I am five -- Sentence Simplification Using Transformers

TL;DR

This paper presents a sentence simplification method using pre-trained transformer models, specifically GPT-2 and BERT, achieving state-of-the-art results without relying on external linguistic resources.

Contribution

The study demonstrates that pure transformer-based models can effectively simplify sentences, surpassing previous methods that used external linguistic databases or control tokens.

Findings

Achieved a SARI score of 46.80 on the Mechanical Turk dataset.

Outperformed previous state-of-the-art results in sentence simplification.

Validated the effectiveness of using only pre-trained transformers for the task.

Abstract

Sentence simplification aims at making the structure of text easier to read and understand while maintaining its original meaning. This can be helpful for people with disabilities, new language learners, or those with low literacy. Simplification often involves removing difficult words and rephrasing the sentence. Previous research have focused on tackling this task by either using external linguistic databases for simplification or by using control tokens for desired fine-tuning of sentences. However, in this paper we purely use pre-trained transformer models. We experiment with a combination of GPT-2 and BERT models, achieving the best SARI score of 46.80 on the Mechanical Turk dataset, which is significantly better than previous state-of-the-art results. The code can be found at https://github.com/amanbasu/sentence-simplification.