Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction

TL;DR

This paper introduces a self-training framework with a pseudo-label scorer to improve aspect sentiment quad prediction, addressing data scarcity and enhancing model performance through quality filtering and large language model-assisted annotation.

Contribution

It proposes a novel self-training method with a pseudo-label scorer and demonstrates its effectiveness and potential for replacing human annotation with large language models.

Findings

Scorer significantly improves self-training effectiveness.

Large language models can replace humans for dataset annotation.

The approach achieves consistent performance gains on public datasets.

Abstract

Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review, which is the most representative and challenging task in aspect-based sentiment analysis. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. To tackle this issue, we propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels, aiming to filter out mismatches and thereby enhance the effectiveness of self-training. We highlight two critical aspects to ensure the scorer's effectiveness and reliability: the quality of the training dataset and its model architecture. To this end, we create a human-annotated comparison dataset and train a generative model on it using ranking-based objectives. Extensive experiments on public ASQP datasets reveal that using our scorer can greatly and consistently improve the effectiveness of self-training. Moreover, we explore the possibility of replacing humans with large language models for comparison dataset annotation, and experiments demonstrate its feasibility. We release our code and data at https://github.com/HITSZ-HLT/ST-w-Scorer-ABSA .