EvolvR: Self-Evolving Pairwise Reasoning for Story Evaluation to Enhance Generation

TL;DR

EvolvR introduces a self-evolving pairwise reasoning framework that improves story evaluation by generating and filtering high-quality reasoning data, leading to state-of-the-art performance and better story generation quality.

Contribution

The paper presents a novel self-evolving framework that synthesizes and filters reasoning data for improved story evaluation and generation, addressing limitations of existing methods.

Findings

Achieves state-of-the-art results on StoryER, HANNA, and OpenMEVA benchmarks.

Effectively enhances story quality when used as a reward model.

Demonstrates robustness and logical rigor through multi-agent data filtering.

Abstract

Although the effectiveness of Large Language Models (LLMs) as judges (LLM-as-a-judge) has been validated, their performance remains limited in open-ended tasks, particularly in story evaluation. Accurate story evaluation is crucial not only for assisting human quality judgment but also for providing key signals to guide story generation. However, existing methods face a dilemma: prompt engineering for closed-source models suffers from poor adaptability, while fine-tuning approaches for open-source models lack the rigorous reasoning capabilities essential for story evaluation. To address this, we propose the Self-Evolving Pairwise Reasoning (EvolvR) framework. Grounded in pairwise comparison, the framework first self-synthesizes score-aligned Chain-of-Thought (CoT) data via a multi-persona strategy. To ensure data quality, these raw CoTs undergo a self-filtering process, utilizing multi-agents to guarantee their logical rigor and robustness. Finally, the evaluator trained on the refined data is deployed as a reward model to guide the story generation task. Experimental results demonstrate that our framework achieves state-of-the-art (SOTA) performance on three evaluation benchmarks including StoryER, HANNA and OpenMEVA. Furthermore, when served as a reward model, it significantly enhances the quality of generated stories, thereby fully validating the superiority of our self-evolving approach.