The Touch\'e23-ValueEval Dataset for Identifying Human Values behind Arguments

TL;DR

This paper introduces the Touch'e23-ValueEval Dataset, a large annotated collection of arguments from diverse sources aimed at detecting underlying human values, enabling improved model training despite increased classification difficulty.

Contribution

The paper presents a new, extensive dataset for identifying human values behind arguments, expanding previous datasets and demonstrating its utility for training better models.

Findings

Larger dataset improves BERT model performance.

Classification difficulty increased with more diverse data.

Dataset covers multiple sources and 54 values.

Abstract

We present the Touch\'e23-ValueEval Dataset for Identifying Human Values behind Arguments. To investigate approaches for the automated detection of human values behind arguments, we collected 9324 arguments from 6 diverse sources, covering religious texts, political discussions, free-text arguments, newspaper editorials, and online democracy platforms. Each argument was annotated by 3 crowdworkers for 54 values. The Touch\'e23-ValueEval dataset extends the Webis-ArgValues-22. In comparison to the previous dataset, the effectiveness of a 1-Baseline decreases, but that of an out-of-the-box BERT model increases. Therefore, though the classification difficulty increased as per the label distribution, the larger dataset allows for training better models.