Exploring the Performance of Large Language Models on Subjective Span Identification Tasks

TL;DR

This paper evaluates how well large language models perform on subjective span identification tasks across sentiment analysis, offensive language detection, and claim verification, highlighting the importance of text relationships.

Contribution

It provides a comprehensive evaluation of LLM strategies like instruction tuning and in-context learning on subjective span identification, an area less explored compared to explicit span tasks.

Findings

Text relationships help LLMs identify spans more accurately

LLMs perform variably across different subjective tasks

Strategies like chain of thought improve span detection

Abstract

Identifying relevant text spans is important for several downstream tasks in NLP, as it contributes to model explainability. While most span identification approaches rely on relatively smaller pre-trained language models like BERT, a few recent approaches have leveraged the latest generation of Large Language Models (LLMs) for the task. Current work has focused on explicit span identification like Named Entity Recognition (NER), while more subjective span identification with LLMs in tasks like Aspect-based Sentiment Analysis (ABSA) has been underexplored. In this paper, we fill this important gap by presenting an evaluation of the performance of various LLMs on text span identification in three popular tasks, namely sentiment analysis, offensive language identification, and claim verification. We explore several LLM strategies like instruction tuning, in-context learning, and chain of thought. Our results indicate underlying relationships within text aid LLMs in identifying precise text spans.