StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples

TL;DR

StyleDistance introduces a novel training method for style embeddings using synthetic paraphrases with controlled style variations, resulting in more content-independent and generalizable style representations that outperform existing models.

Contribution

We propose a new approach using synthetic data and contrastive learning to improve content independence in style embeddings.

Findings

Enhanced content-independence of style embeddings.

Outperforms existing style representations on benchmarks.

Effective in downstream style transfer tasks.

Abstract

Style representations aim to embed texts with similar writing styles closely and texts with different styles far apart, regardless of content. However, the contrastive triplets often used for training these representations may vary in both style and content, leading to potential content leakage in the representations. We introduce StyleDistance, a novel approach to training stronger content-independent style embeddings. We use a large language model to create a synthetic dataset of near-exact paraphrases with controlled style variations, and produce positive and negative examples across 40 distinct style features for precise contrastive learning. We assess the quality of our synthetic data and embeddings through human and automatic evaluations. StyleDistance enhances the content-independence of style embeddings, which generalize to real-world benchmarks and outperform leading style representations in downstream applications. Our model can be found at https://huggingface.co/StyleDistance/styledistance .