A Multi-Task Text Classification Pipeline with Natural Language Explanations: A User-Centric Evaluation in Sentiment Analysis and Offensive Language Identification in Greek Tweets

TL;DR

This paper presents a user-centric pipeline for text classification that generates natural language explanations, demonstrated on sentiment analysis and offensive language detection in Greek tweets, enhancing interpretability for end-users.

Contribution

It introduces a novel multi-task pipeline combining classification and explanation generation in natural language, adaptable to various text classification tasks with available ground truth rationales.

Findings

User study shows promising interpretability improvements

Effective explanations generated for Greek sentiment and offensive language tasks

Pipeline adaptable to different text classification problems

Abstract

Interpretability is a topic that has been in the spotlight for the past few years. Most existing interpretability techniques produce interpretations in the form of rules or feature importance. These interpretations, while informative, may be harder to understand for non-expert users and therefore, cannot always be considered as adequate explanations. To that end, explanations in natural language are often preferred, as they are easier to comprehend and also more presentable to end-users. This work introduces an early concept for a novel pipeline that can be used in text classification tasks, offering predictions and explanations in natural language. It comprises of two models: a classifier for labelling the text and an explanation generator which provides the explanation. The proposed pipeline can be adopted by any text classification task, given that ground truth rationales are available to train the explanation generator. Our experiments are centred around the tasks of sentiment analysis and offensive language identification in Greek tweets, using a Greek Large Language Model (LLM) to obtain the necessary explanations that can act as rationales. The experimental evaluation was performed through a user study based on three different metrics and achieved promising results for both datasets.