Emissions and Performance Trade-off Between Small and Large Language Models

TL;DR

This paper compares the environmental impact and performance of small versus large language models, showing that smaller models can achieve similar results with significantly lower carbon emissions in many tasks.

Contribution

It provides a comparative analysis demonstrating that fine-tuned small language models can match large models' performance while reducing emissions, promoting sustainable AI practices.

Findings

SLMs maintain comparable performance in 4 out of 6 tasks

Significant reduction in carbon emissions during inference with SLMs

Supports the viability of smaller models for sustainable AI

Abstract

The advent of Large Language Models (LLMs) has raised concerns about their enormous carbon footprint, starting with energy-intensive training and continuing through repeated inference. This study investigates the potential of using fine-tuned Small Language Models (SLMs) as a sustainable alternative for predefined tasks. Here, we present a comparative analysis of the performance-emissions trade-off between LLMs and fine-tuned SLMs across selected tasks under Natural Language Processing, Reasoning and Programming. Our results show that in four out of the six selected tasks, SLMs maintained comparable performances for a significant reduction in carbon emissions during inference. Our findings demonstrate the viability of smaller models in mitigating the environmental impact of resource-heavy LLMs, thus advancing towards sustainable, green AI.