An Empirical Study on Measuring the Similarity of Sentential Arguments with Language Model Domain Adaptation

TL;DR

This study explores using domain-adapted language models to measure sentential argument similarity, reducing reliance on labeled data and improving correlation with human judgments in argument mining.

Contribution

It demonstrates that domain-adapted language models can effectively measure argument similarity with less labeled data, advancing argument mining techniques.

Findings

Improved correlation with human similarity scores

Achieved comparable performance with 60% of labeled data

Enhanced unsupervised argument similarity measurement

Abstract

Measuring the similarity between two different sentential arguments is an important task in argument mining. However, one of the challenges in this field is that the dataset must be annotated using expertise in a variety of topics, making supervised learning with labeled data expensive. In this paper, we investigated whether this problem could be alleviated through transfer learning. We first adapted a pretrained language model to a domain of interest using self-supervised learning. Then, we fine-tuned the model to a task of measuring the similarity between sentences taken from different domains. Our approach improves a correlation with human-annotated similarity scores compared to competitive baseline models on the Argument Facet Similarity dataset in an unsupervised setting. Moreover, we achieve comparable performance to a fully supervised baseline model by using only about 60% of the labeled data samples. We believe that our work suggests the possibility of a generalized argument clustering model for various argumentative topics.