# TSTBench: A Comprehensive Benchmark for Text Style Transfer

## TL;DR

TSTBench is a new benchmark for evaluating text style transfer methods, offering a codebase and standardized protocol to improve evaluation consistency and reproducibility.

## Contribution

The novel contribution is TSTBench, a comprehensive benchmark with implementations of 13 algorithms and a standardized protocol for text style transfer.

## Key findings

- TSTBench includes a codebase with 13 state-of-the-art algorithms and a standardized protocol for text style transfer.
- Experiments across seven datasets produced over 7000 evaluations, providing insights into TST performance and evaluation processes.

## Abstract

In recent years, researchers in computational linguistics have shown a growing interest in the style of text, with a specific focus on the text style transfer (TST) task. While numerous innovative methods have been proposed, it has been observed that the existing evaluations are insufficient to validate the claims and precisely measure the performance. This challenge primarily stems from rapid advancements and diverse settings of these methods, with the associated (re)implementation and reproducibility hurdles. To bridge this gap, we introduce a comprehensive benchmark for TST known as TSTBench. TSTBench includes a codebase encompassing implementations of 13 state-of-the-art algorithms and a standardized protocol for text style transfer. Based on the codebase and protocol, we have conducted thorough experiments across seven datasets, resulting in a total of 7000+ evaluations. Our work provides extensive analysis from various perspectives, explores the performance of representative baselines across various datasets, and offers insights into the task and evaluation processes to guide future research in TST.