R2-Write: Reflection and Revision for Open-Ended Writing with Deep Reasoning

TL;DR

This paper introduces R2-Write, a framework enhancing open-ended writing with reflection and revision patterns, leading to significant performance improvements in creative and research writing tasks.

Contribution

It presents a novel automated framework that synthesizes high-quality thinking trajectories with explicit reflection and revision, addressing limitations of existing reasoning models.

Findings

R2-Write significantly improves open-ended writing performance.

Explicit reflection and revision patterns enhance deep reasoning.

The process reward mechanism improves token efficiency.

Abstract

While deep reasoning with long chain-of-thought has dramatically improved large language models in verifiable domains like mathematics, its effectiveness for open-ended tasks such as writing remains unexplored. In this paper, we conduct a systematic investigation revealing that existing mainstream reasoning models achieve limited gains on open-ended writing tasks. Our further analysis shows that these models lack deep reflection and revision patterns in open-ended writing, resulting in substantially smaller improvements compared to mathematical reasoning tasks. To address this limitation, we introduce R2-Write: an automated framework that synthesizes high-quality thinking trajectories enriched with explicit reflection and revision patterns through iterative writer-judge interaction. To prevent redundant reflections, we design a process reward mechanism that supervises reflection quality during reinforcement learning, improving both performance and token efficiency. Extensive experiments across multiple creative writing and deep-research benchmarks demonstrate significant improvements, validating that explicitly incorporating reflection and revision patterns unlocks deep reasoning capabilities for open-ended writing tasks.