Who is Addressed in this Comment? Automatically Classifying Meta-Comments in News Comments

TL;DR

This paper presents an automated method to identify and classify meta-comments in online news comments, helping newsrooms manage large volumes of user feedback more effectively.

Contribution

It introduces and compares feature-based and end-to-end learning classifiers for meta-comment detection in news comments, with evaluations on German and Austrian news datasets.

Findings

Achieved $F_{0.5}$ scores between 76% and 91%.

Identified key features for meta-comment classification.

Enhanced understanding of constructive participation in online journalism.

Abstract

User comments have become an essential part of online journalism. However, newsrooms are often overwhelmed by the vast number of diverse comments, for which a manual analysis is barely feasible. Identifying meta-comments that address or mention newsrooms, individual journalists, or moderators and that may call for reactions is particularly critical. In this paper, we present an automated approach to identify and classify meta-comments. We compare comment classification based on manually extracted features with an end-to-end learning approach. We develop, optimize, and evaluate multiple classifiers on a comment dataset of the large German online newsroom SPIEGEL Online and the 'One Million Posts' corpus of DER STANDARD, an Austrian newspaper. Both optimized classification approaches achieved encouraging $F_{0.5}$ values between 76% and 91%. We report on the most significant classification features with the results of a qualitative analysis and discuss how our work contributes to making participation in online journalism more constructive.