Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style

TL;DR

This study investigates whether users can effectively modify LLM-generated drafts to match their personal style through post-editing, revealing partial success and persistent stylistic traces.

Contribution

It provides empirical evidence on the extent to which post-editing can adapt LLM outputs to individual styles and highlights the gap between perceived authenticity and measurable stylistic similarity.

Findings

Post-editing increases stylistic similarity to unassisted writing.

Post-edited text remains closer in style to LLM output than to original human writing.

Participants perceive post-edited text as more authentic than what metrics suggest.

Abstract

Despite the growing use of large language models (LLMs) for writing tasks, users may hesitate to rely on LLMs when personal style is important. Post-editing LLM-generated drafts or translations is a common collaborative writing strategy, but it remains unclear whether users can effectively reshape LLM-generated text to reflect their personal style. We conduct a pre-registered online study ($n=81$) in which participants post-edit LLM-generated drafts for writing tasks where personal style matters to them. Using embedding-based style similarity metrics, we find that post-editing increases stylistic similarity to participants' unassisted writing and reduces similarity to fully LLM-generated output. However, post-edited text still remains stylistically closer in style to LLM text than to participants' unassisted control text, and it exhibits reduced stylistic diversity compared to unassisted human text. We find a gap between perceived stylistic authenticity and model-measured stylistic similarity, with post-edited text often perceived as representative of participants' personal style despite remaining detectable LLM stylistic traces.