KE-QI: A Knowledge Enhanced Article Quality Identification Dataset

TL;DR

This paper introduces KE-QI, a new dataset for identifying high-quality articles by leveraging external knowledge, and proposes a model that fuses article text with knowledge for improved classification.

Contribution

The paper creates the KE-QI dataset linking articles to external knowledge and develops a novel model that integrates external knowledge with text for article quality assessment.

Findings

The KE-QI dataset contains 10,000 annotated articles.

The proposed model achieves approximately 78% F1 score.

External knowledge integration improves quality classification performance.

Abstract

With so many articles of varying qualities being produced every moment, it is a very urgent task to screen outstanding articles and commit them to social media. To our best knowledge, there is a lack of datasets and mature research works in identifying high-quality articles. Consequently, we conduct some surveys and finalize 7 objective indicators to annotate the quality of 10k articles. During annotation, we find that many characteristics of high-quality articles (e.g., background) rely more on extensive external knowledge than inner semantic information of articles. In response, we link extracted article entities to Baidu Encyclopedia, then propose Knowledge Enhanced article Quality Identification (KE-QI) dataset. To make better use of external knowledge, we propose a compound model which fuses the text and external knowledge information via a gate unit to classify the quality of an article. Our experimental results on KE-QI show that with initialization of our pre-trained Node2Vec model, our model achieves about 78\% $F_1$, outperforming other baselines.