Contrastive Learning enhanced Author-Style Headline Generation

TL;DR

This paper introduces CLH3G, a novel Seq2Seq model that leverages historical headlines and contrastive learning to generate more stylistically consistent and accurate headlines for articles.

Contribution

It proposes a new model integrating historical headlines and contrastive learning to capture author style, improving headline generation quality.

Findings

Historical headlines significantly improve headline relevance.

Contrastive learning enhances stylistic feature learning.

The proposed methods outperform baseline models.

Abstract

Headline generation is a task of generating an appropriate headline for a given article, which can be further used for machine-aided writing or enhancing the click-through ratio. Current works only use the article itself in the generation, but have not taken the writing style of headlines into consideration. In this paper, we propose a novel Seq2Seq model called CLH3G (Contrastive Learning enhanced Historical Headlines based Headline Generation) which can use the historical headlines of the articles that the author wrote in the past to improve the headline generation of current articles. By taking historical headlines into account, we can integrate the stylistic features of the author into our model, and generate a headline not only appropriate for the article, but also consistent with the author's style. In order to efficiently learn the stylistic features of the author, we further introduce a contrastive learning based auxiliary task for the encoder of our model. Besides, we propose two methods to use the learned stylistic features to guide both the pointer and the decoder during the generation. Experimental results show that historical headlines of the same user can improve the headline generation significantly, and both the contrastive learning module and the two style features fusion methods can further boost the performance.