A Comparative Study of Task Adaptation Techniques of Large Language Models for Identifying Sustainable Development Goals

TL;DR

This paper compares different large language models and adaptation techniques for classifying texts related to Sustainable Development Goals, showing smaller models can match larger ones with proper prompt engineering.

Contribution

It evaluates the effectiveness of various task adaptation methods for LLMs in SDG text classification, highlighting the potential of smaller models with prompt optimization.

Findings

Smaller models perform comparably to larger models with prompt engineering.

Zero-Shot and Few-Shot learning are effective for SDG classification.

Fine-tuning improves model performance in SDG text classification.

Abstract

In 2012, the United Nations introduced 17 Sustainable Development Goals (SDGs) aimed at creating a more sustainable and improved future by 2030. However, tracking progress toward these goals is difficult because of the extensive scale and complexity of the data involved. Text classification models have become vital tools in this area, automating the analysis of vast amounts of text from a variety of sources. Additionally, large language models (LLMs) have recently proven indispensable for many natural language processing tasks, including text classification, thanks to their ability to recognize complex linguistic patterns and semantics. This study analyzes various proprietary and open-source LLMs for a single-label, multi-class text classification task focused on the SDGs. Then, it also evaluates the effectiveness of task adaptation techniques (i.e., in-context learning approaches), namely Zero-Shot and Few-Shot Learning, as well as Fine-Tuning within this domain. The results reveal that smaller models, when optimized through prompt engineering, can perform on par with larger models like OpenAI's GPT (Generative Pre-trained Transformer).