Fine-grained Text Style Transfer with Diffusion-Based Language Models

TL;DR

This paper introduces a diffusion-based language model trained on StylePTB for fine-grained text style transfer, achieving state-of-the-art results without external knowledge and demonstrating strong performance in low-resource settings.

Contribution

The work presents a diffusion-based model that outperforms previous methods on StylePTB, highlighting its effectiveness for fine-grained style transfer without relying on pre-trained resources.

Findings

Achieved state-of-the-art performance on StylePTB

Outperformed models using external knowledge

Effective in low-resource scenarios

Abstract

Diffusion probabilistic models have shown great success in generating high-quality images controllably, and researchers have tried to utilize this controllability into text generation domain. Previous works on diffusion-based language models have shown that they can be trained without external knowledge (such as pre-trained weights) and still achieve stable performance and controllability. In this paper, we trained a diffusion-based model on StylePTB dataset, the standard benchmark for fine-grained text style transfers. The tasks in StylePTB requires much more refined control over the output text compared to tasks evaluated in previous works, and our model was able to achieve state-of-the-art performance on StylePTB on both individual and compositional transfers. Moreover, our model, trained on limited data from StylePTB without external knowledge, outperforms previous works that utilized pretrained weights, embeddings, and external grammar parsers, and this may indicate that diffusion-based language models have great potential under low-resource settings.