Towards Few-Shot Identification of Morality Frames using In-Context Learning

TL;DR

This paper explores using large language models with few-shot prompting to identify morality frames in text, aiming to reduce reliance on costly human annotations for socio-linguistic concept detection.

Contribution

It introduces prompting-based methods with LLMs for morality frame identification, demonstrating effectiveness compared to few-shot RoBERTa.

Findings

Promising results in morality frame detection using LLMs

Effective few-shot performance with minimal exemplars

Potential to reduce annotation costs in socio-linguistic NLP

Abstract

Data scarcity is a common problem in NLP, especially when the annotation pertains to nuanced socio-linguistic concepts that require specialized knowledge. As a result, few-shot identification of these concepts is desirable. Few-shot in-context learning using pre-trained Large Language Models (LLMs) has been recently applied successfully in many NLP tasks. In this paper, we study few-shot identification of a psycho-linguistic concept, Morality Frames (Roy et al., 2021), using LLMs. Morality frames are a representation framework that provides a holistic view of the moral sentiment expressed in text, identifying the relevant moral foundation (Haidt and Graham, 2007) and at a finer level of granularity, the moral sentiment expressed towards the entities mentioned in the text. Previous studies relied on human annotation to identify morality frames in text which is expensive. In this paper, we propose prompting-based approaches using pretrained Large Language Models for identification of morality frames, relying only on few-shot exemplars. We compare our models' performance with few-shot RoBERTa and found promising results.