Preference-based Interactive Multi-Document Summarisation

TL;DR

This paper introduces APRIL, an active preference-based reinforcement learning framework for multi-document summarisation that significantly reduces user interaction rounds and outperforms existing methods.

Contribution

The paper proposes APRIL, combining active learning, preference learning, and neural reinforcement learning to improve interactive summarisation efficiency and effectiveness.

Findings

APRIL requires fewer interaction rounds than previous methods.

Users can reliably provide preferences over summaries.

APRIL outperforms state-of-the-art in both simulations and real-user tests.

Abstract

Interactive NLP is a promising paradigm to close the gap between automatic NLP systems and the human upper bound. Preference-based interactive learning has been successfully applied, but the existing methods require several thousand interaction rounds even in simulations with perfect user feedback. In this paper, we study preference-based interactive summarisation. To reduce the number of interaction rounds, we propose the Active Preference-based ReInforcement Learning (APRIL) framework. APRIL uses Active Learning to query the user, Preference Learning to learn a summary ranking function from the preferences, and neural Reinforcement Learning to efficiently search for the (near-)optimal summary. Our results show that users can easily provide reliable preferences over summaries and that APRIL outperforms the state-of-the-art preference-based interactive method in both simulation and real-user experiments.