Generating News Headlines with Recurrent Neural Networks

TL;DR

This paper presents a neural network model with attention mechanisms for generating news headlines from article text, demonstrating effective paraphrasing and insights into attention functions, with a simpler model outperforming complex ones.

Contribution

It introduces a recurrent neural network with simplified attention for headline generation, showing improved performance and interpretability over more complex attention models.

Findings

Simplified attention mechanism outperforms complex models.

Neural network effectively paraphrases news articles.

Analysis reveals neuron functions in attention mechanism.

Abstract

We describe an application of an encoder-decoder recurrent neural network with LSTM units and attention to generating headlines from the text of news articles. We find that the model is quite effective at concisely paraphrasing news articles. Furthermore, we study how the neural network decides which input words to pay attention to, and specifically we identify the function of the different neurons in a simplified attention mechanism. Interestingly, our simplified attention mechanism performs better that the more complex attention mechanism on a held out set of articles.