Transductive Learning for Unsupervised Text Style Transfer

TL;DR

This paper introduces a transductive learning method for unsupervised text style transfer that leverages retrieval-based context-aware style representations to improve consistency and performance.

Contribution

It proposes a novel transductive approach using retrieval-augmented style embeddings, addressing limitations of inductive methods in unsupervised style transfer.

Findings

Outperforms several strong baseline models

Effective in reducing style inconsistency issues

Demonstrates general applicability to unsupervised style transfer

Abstract

Unsupervised style transfer models are mainly based on an inductive learning approach, which represents the style as embeddings, decoder parameters, or discriminator parameters and directly applies these general rules to the test cases. However, the lacking of parallel corpus hinders the ability of these inductive learning methods on this task. As a result, it is likely to cause severe inconsistent style expressions, like `the salad is rude`. To tackle this problem, we propose a novel transductive learning approach in this paper, based on a retrieval-based context-aware style representation. Specifically, an attentional encoder-decoder with a retriever framework is utilized. It involves top-K relevant sentences in the target style in the transfer process. In this way, we can learn a context-aware style embedding to alleviate the above inconsistency problem. In this paper, both sparse (BM25) and dense retrieval functions (MIPS) are used, and two objective functions are designed to facilitate joint learning. Experimental results show that our method outperforms several strong baselines. The proposed transductive learning approach is general and effective to the task of unsupervised style transfer, and we will apply it to the other two typical methods in the future.