Studying the role of named entities for content preservation in text style transfer

TL;DR

This paper investigates the importance of named entities in maintaining content integrity during formality style transfer in task-oriented dialogue domains, introducing a new dataset and a method to improve content preservation.

Contribution

It introduces a new dataset for evaluating content similarity in style transfer and proposes a technique leveraging named entities to enhance content preservation.

Findings

Named entities are crucial for content preservation in style transfer.

A new dataset from task-oriented dialogues aids in evaluating style transfer models.

Using named entity information improves content similarity measures.

Abstract

Text style transfer techniques are gaining popularity in Natural Language Processing, finding various applications such as text detoxification, sentiment, or formality transfer. However, the majority of the existing approaches were tested on such domains as online communications on public platforms, music, or entertainment yet none of them were applied to the domains which are typical for task-oriented production systems, such as personal plans arrangements (e.g. booking of flights or reserving a table in a restaurant). We fill this gap by studying formality transfer in this domain. We noted that the texts in this domain are full of named entities, which are very important for keeping the original sense of the text. Indeed, if for example, someone communicates the destination city of a flight it must not be altered. Thus, we concentrate on the role of named entities in content preservation for formality text style transfer. We collect a new dataset for the evaluation of content similarity measures in text style transfer. It is taken from a corpus of task-oriented dialogues and contains many important entities related to realistic requests that make this dataset particularly useful for testing style transfer models before using them in production. Besides, we perform an error analysis of a pre-trained formality transfer model and introduce a simple technique to use information about named entities to enhance the performance of baseline content similarity measures used in text style transfer.