AnnoABSA: A Web-Based Annotation Tool for Aspect-Based Sentiment Analysis with Retrieval-Augmented Suggestions

TL;DR

AnnoABSA is a customizable web-based tool for Aspect-Based Sentiment Analysis that integrates retrieval-augmented LLM suggestions to enhance annotation quality through a human-in-the-loop approach.

Contribution

It introduces the first web-based ABSA annotation tool with integrated LLM-based suggestions and a retrieval mechanism for improving annotation accuracy over time.

Findings

Supports full spectrum of ABSA tasks

Provides context-aware LLM suggestions

Improves prediction quality with retrieval of similar examples

Abstract

We introduce AnnoABSA, the first web-based annotation tool to support the full spectrum of Aspect-Based Sentiment Analysis (ABSA) tasks. The tool is highly customizable, enabling flexible configuration of sentiment elements and task-specific requirements. Alongside manual annotation, AnnoABSA provides optional Large Language Model (LLM)-based retrieval-augmented generation (RAG) suggestions that offer context-aware assistance in a human-in-the-loop approach, keeping the human annotator in control. To improve prediction quality over time, the system retrieves the ten most similar examples that are already annotated and adds them as few-shot examples in the prompt, ensuring that suggestions become increasingly accurate as the annotation process progresses. Released as open-source software under the MIT License, AnnoABSA is freely accessible and easily extendable for research and practical applications.