Improving (Dis)agreement Detection with Inductive Social Relation Information From Comment-Reply Interactions

TL;DR

This paper introduces a novel approach that combines textual analysis with inductively extracted social relation graphs from comment-reply data to improve (dis)agreement detection across domains, achieving state-of-the-art results.

Contribution

It proposes a method to incorporate social relation information from comment-reply interactions into (dis)agreement detection, enhancing performance especially in cross-domain scenarios.

Findings

Social relations boost (dis)agreement detection accuracy.

The model performs well on in-domain and cross-domain tasks.

Incorporating social relation graphs improves detection of long-token comments.

Abstract

(Dis)agreement detection aims to identify the authors' attitudes or positions (\textit{{agree, disagree, neutral}}) towards a specific text. It is limited for existing methods merely using textual information for identifying (dis)agreements, especially for cross-domain settings. Social relation information can play an assistant role in the (dis)agreement task besides textual information. We propose a novel method to extract such relation information from (dis)agreement data into an inductive social relation graph, merely using the comment-reply pairs without any additional platform-specific information. The inductive social relation globally considers the historical discussion and the relation between authors. Textual information based on a pre-trained language model and social relation information encoded by pre-trained RGCN are jointly considered for (dis)agreement detection. Experimental results show that our model achieves state-of-the-art performance for both the in-domain and cross-domain tasks on the benchmark -- DEBAGREEMENT. We find social relations can boost the performance of the (dis)agreement detection model, especially for the long-token comment-reply pairs, demonstrating the effectiveness of the social relation graph. We also explore the effect of the knowledge graph embedding methods, the information fusing method, and the time interval in constructing the social relation graph, which shows the effectiveness of our model.