UniHD at TSAR-2022 Shared Task: Is Compute All We Need for Lexical Simplification?

TL;DR

This paper demonstrates that a simple ensemble of GPT-3 prompts can outperform complex lexical simplification models, achieving state-of-the-art results in English and transferability to Spanish and Portuguese.

Contribution

It introduces a minimalistic GPT-3 prompt ensemble approach for lexical simplification, showing competitive performance and effective language transfer techniques.

Findings

GPT-3 ensemble surpasses complex models in English

Language transfer enables state-of-the-art results in Spanish and Portuguese

Prompting strategies significantly impact simplification quality

Abstract

Previous state-of-the-art models for lexical simplification consist of complex pipelines with several components, each of which requires deep technical knowledge and fine-tuned interaction to achieve its full potential. As an alternative, we describe a frustratingly simple pipeline based on prompted GPT-3 responses, beating competing approaches by a wide margin in settings with few training instances. Our best-performing submission to the English language track of the TSAR-2022 shared task consists of an ``ensemble'' of six different prompt templates with varying context levels. As a late-breaking result, we further detail a language transfer technique that allows simplification in languages other than English. Applied to the Spanish and Portuguese subset, we achieve state-of-the-art results with only minor modification to the original prompts. Aside from detailing the implementation and setup, we spend the remainder of this work discussing the particularities of prompting and implications for future work. Code for the experiments is available online at https://github.com/dennlinger/TSAR-2022-Shared-Task