CUT: Controllable Unsupervised Text Simplification

TL;DR

This paper introduces two unsupervised methods for controllable text simplification, enabling the generation of simpler or more complex texts without labeled data, and demonstrates competitive results on benchmark datasets.

Contribution

It proposes novel unsupervised mechanisms, back translation with control tokens and simplicity-aware beam search, for controlling text complexity in simplification tasks.

Findings

Achieved a SARI score of 46.88% on Newsela.

Attained a FKGL of 3.65% on Newsela.

Demonstrated competitive performance compared to supervised methods.

Abstract

In this paper, we focus on the challenge of learning controllable text simplifications in unsupervised settings. While this problem has been previously discussed for supervised learning algorithms, the literature on the analogies in unsupervised methods is scarse. We propose two unsupervised mechanisms for controlling the output complexity of the generated texts, namely, back translation with control tokens (a learning-based approach) and simplicity-aware beam search (decoding-based approach). We show that by nudging a back-translation algorithm to understand the relative simplicity of a text in comparison to its noisy translation, the algorithm self-supervises itself to produce the output of the desired complexity. This approach achieves competitive performance on well-established benchmarks: SARI score of 46.88% and FKGL of 3.65% on the Newsela dataset.