Multi-label Classification of User Reactions in Online News

TL;DR

This paper evaluates multi-label classification methods for detecting user reactions in online news, introducing a new Brazilian Portuguese news corpus and analyzing various algorithms' effectiveness.

Contribution

It presents a new annotated news corpus in Brazilian Portuguese and compares multiple multi-label classification techniques for reaction detection.

Findings

Classifier Chains with Random Forest achieved most correct predictions.

Binary Relevance with LSTM and Random Forest performed best considering class distribution.

Extensive tests demonstrate the effectiveness of different problem transformation methods.

Abstract

The increase in the number of Internet users and the strong interaction brought by Web 2.0 made the Opinion Mining an important task in the area of natural language processing. Although several methods are capable of performing this task, few use multi-label classification, where there is a group of true labels for each example. This type of classification is useful for situations where the opinions are analyzed from the perspective of the reader, this happens because each person can have different interpretations and opinions on the same subject. This paper discuss the efficiency of problem transformation methods combined with different classification algorithms for the task of multi-label classification of reactions in news texts. To do that, extensive tests were carried out on two news corpora written in Brazilian Portuguese annotated with reactions. A new corpus called BFRC-PT is presented. In the tests performed, the highest number of correct predictions was obtained with the Classifier Chains method combined with the Random Forest algorithm. When considering the class distribution, the best results were obtained with the Binary Relevance method combined with the LSTM and Random Forest algorithms.