Text Style Transfer with Machine Translation for Graphic Designs

TL;DR

This paper investigates methods to improve word alignment accuracy in machine translation to preserve text styling in graphic design translations, comparing NMT, LLM, and hybrid approaches.

Contribution

It introduces three novel methods leveraging NMT and LLM technologies for better word alignment in style transfer, and evaluates their performance against attention-based baselines.

Findings

Attention head approach outperforms NMT and LLM in alignment accuracy.

Hybrid NMT+LLM approach achieves comparable results to the attention baseline.

Proposed methods utilize commercially available translation technologies.

Abstract

Globalization of graphic designs such as those used in marketing materials and magazines is increasingly important for communication to broad audiences. To accomplish this, the textual content in the graphic designs needs to be accurately translated and have the text styling preserved in order to fit visually into the design. Preserving text styling requires high accuracy word alignment between the original and the translated text. The problem of word alignment between source and translated text is long known. The industry standards for extracting word alignments are defined by Giza++ and attention probabilities from neural machine translation (NMT) models. In this paper, we explore three new methods to tackle the word alignment problem for transferring text styles from the source to the translated text. The proposed methods are developed on top of commercially available NMT and LLM translation technologies. They include: NMT with custom input and output tags for text styling; LLM with custom input and output tags; a hybrid with NMT for translation followed by an LLM with use of unigram mappings. To analyze the performance of these solutions, their alignment results are compared with the results of an attention head approach to gauge their usability in graphic design applications. Interestingly, the attention head strong baseline proves more accurate than the LLM or NMT approach and on par with the hybrid NMT+LLM approach.