Learning to Determine the Quality of News Headlines

TL;DR

This paper introduces a deep learning model that assesses news headline quality by analyzing click data and semantic relations, aiming to improve online user engagement and reduce misleading headlines.

Contribution

It proposes four novel quality indicators based on website logs and a deep learning approach that considers headline-body semantics for predicting headline quality.

Findings

Model outperforms existing NLP methods

Effective use of click and dwell time data

Improves headline quality assessment accuracy

Abstract

Today, most newsreaders read the online version of news articles rather than traditional paper-based newspapers. Also, news media publishers rely heavily on the income generated from subscriptions and website visits made by newsreaders. Thus, online user engagement is a very important issue for online newspapers. Much effort has been spent on writing interesting headlines to catch the attention of online users. On the other hand, headlines should not be misleading (e.g., clickbaits); otherwise, readers would be disappointed when reading the content. In this paper, we propose four indicators to determine the quality of published news headlines based on their click count and dwell time, which are obtained by website log analysis. Then, we use soft target distribution of the calculated quality indicators to train our proposed deep learning model which can predict the quality of unpublished news headlines. The proposed model not only processes the latent features of both headline and body of the article to predict its headline quality but also considers the semantic relation between headline and body as well. To evaluate our model, we use a real dataset from a major Canadian newspaper. Results show our proposed model outperforms other state-of-the-art NLP models.