Human-Centered Supervision for Sentiment Analysis in Telugu: A Systematic Inquiry Beyond Accuracy

TL;DR

This paper introduces TeSent, a large Telugu sentiment dataset with human rationales, and demonstrates that rationale-based supervision improves model alignment, performance, and fairness in low-resource sentiment analysis.

Contribution

It presents a new Telugu sentiment dataset with human rationales and evaluates transformer models with rationale supervision, advancing interpretability and fairness in low-resource NLP.

Findings

Rationale supervision improves model alignment with human reasoning.

Incorporating human rationales enhances predictive performance.

The study provides insights into fairness and explanation quality in low-resource settings.

Abstract

Sentiment analysis for low-resource languages remains challenging in an era where interpretability, human alignment, and fairness are increasingly non-negotiable aspects of modern machine learning systems. These challenges stem both from the scarcity of annotated data and from the resulting difficulty of conducting reliable, human-interpretable analyses that go beyond predictive accuracy. Telugu, one of the primary Dravidian languages with over 96 million speakers, is not an exception. In this work, we first introduce TeSent, a large-scale Telugu sentiment classification dataset annotated with sentiment labels and human-selected rationales from multiple native speakers. This resource enables the study of rationale-based supervision for aligning models with human reasoning in this low-resource setting. We fine-tune five transformer-based models with and without rationale supervision and evaluate them on classification performance, explanation quality, and social bias. To facilitate controlled fairness evaluation, we additionally construct TeEEC, an evaluation corpus for Telugu sentiment analysis. Our results show that incorporating human rationales consistently improves alignment and often leads to holistic gains in predictive performance. We further provide extensive analysis of multi-facade explanation quality and fairness, offering insights into the broader effects of alignment-oriented supervision in resource-scarce language contexts.