Unsupervised Editing for Counterfactual Stories

TL;DR

This paper introduces EDUCAT, an unsupervised editing approach for counterfactual story rewriting that balances logical coherence with minimal edits, outperforming existing methods on a public benchmark.

Contribution

EDUCAT is a novel unsupervised method that detects target positions based on causal effects and generates stories with minimal edits, improving trade-offs in counterfactual story rewriting.

Findings

EDUCAT achieves the best trade-off between coherence and minimal edits.

It outperforms state-of-the-art unsupervised methods on benchmark evaluations.

Human evaluations confirm its effectiveness in maintaining story quality.

Abstract

Creating what-if stories requires reasoning about prior statements and possible outcomes of the changed conditions. One can easily generate coherent endings under new conditions, but it would be challenging for current systems to do it with minimal changes to the original story. Therefore, one major challenge is the trade-off between generating a logical story and rewriting with minimal-edits. In this paper, we propose EDUCAT, an editing-based unsupervised approach for counterfactual story rewriting. EDUCAT includes a target position detection strategy based on estimating causal effects of the what-if conditions, which keeps the causal invariant parts of the story. EDUCAT then generates the stories under fluency, coherence and minimal-edits constraints. We also propose a new metric to alleviate the shortcomings of current automatic metrics and better evaluate the trade-off. We evaluate EDUCAT on a public counterfactual story rewriting benchmark. Experiments show that EDUCAT achieves the best trade-off over unsupervised SOTA methods according to both automatic and human evaluation. The resources of EDUCAT are available at: https://github.com/jiangjiechen/EDUCAT.