Neural Headline Generation with Sentence-wise Optimization

TL;DR

This paper introduces a sentence-level optimization approach for neural headline generation using minimum risk training, leading to significant performance improvements over existing models on English and Chinese datasets.

Contribution

It proposes a novel sentence-level training method with minimum risk training for neural headline generation, surpassing traditional word-level maximum likelihood approaches.

Findings

Outperforms state-of-the-art systems on English headline generation

Achieves significant improvements on Chinese headline tasks

Demonstrates effectiveness of sentence-level optimization

Abstract

Recently, neural models have been proposed for headline generation by learning to map documents to headlines with recurrent neural networks. Nevertheless, as traditional neural network utilizes maximum likelihood estimation for parameter optimization, it essentially constrains the expected training objective within word level rather than sentence level. Moreover, the performance of model prediction significantly relies on training data distribution. To overcome these drawbacks, we employ minimum risk training strategy in this paper, which directly optimizes model parameters in sentence level with respect to evaluation metrics and leads to significant improvements for headline generation. Experiment results show that our models outperforms state-of-the-art systems on both English and Chinese headline generation tasks.