A Better Choice: Entire-space Datasets for Aspect Sentiment Triplet Extraction

TL;DR

This paper advocates for using the entire-space dataset version in aspect sentiment triplet extraction to better reflect real-world scenarios and improve evaluation accuracy.

Contribution

The paper analyzes dataset versions for ASTE and proposes using the entire-space version for more realistic and consistent model evaluation.

Findings

Evaluating on non-entire-space datasets inflates performance metrics.

Models trained on the entire-space dataset perform better.

Entire-space datasets include non-triplet sentences, aligning with real-world data.

Abstract

Aspect sentiment triplet extraction (ASTE) aims to extract aspect term, sentiment and opinion term triplets from sentences. Since the initial datasets used to evaluate models on ASTE had flaws, several studies later corrected the initial datasets and released new versions of the datasets independently. As a result, different studies select different versions of datasets to evaluate their methods, which makes ASTE-related works hard to follow. In this paper, we analyze the relation between different versions of datasets and suggest that the entire-space version should be used for ASTE. Besides the sentences containing triplets and the triplets in the sentences, the entire-space version additionally includes the sentences without triplets and the aspect terms which do not belong to any triplets. Hence, the entire-space version is consistent with real-world scenarios and evaluating models on the entire-space version can better reflect the models' performance in real-world scenarios. In addition, experimental results show that evaluating models on non-entire-space datasets inflates the performance of existing models and models trained on the entire-space version can obtain better performance.