Performance in a dialectal profiling task of LLMs for varieties of Brazilian Portuguese

TL;DR

This study investigates how large language models recognize and discriminate between dialectal varieties of Brazilian Portuguese, revealing biases and sociolinguistic considerations in their responses.

Contribution

It provides an analysis of dialectal biases in four major LLMs using prompt engineering, contributing sociolinguistic insights for more equitable NLP technology.

Findings

LLMs show varying degrees of dialectal discrimination.

Prompt engineering can reveal underlying biases.

Sociolinguistic rules influence model responses.

Abstract

Different of biases are reproduced in LLM-generated responses, including dialectal biases. A study based on prompt engineering was carried out to uncover how LLMs discriminate varieties of Brazilian Portuguese, specifically if sociolinguistic rules are taken into account in four LLMs: GPT 3.5, GPT-4o, Gemini, and Sabi.-2. The results offer sociolinguistic contributions for an equity fluent NLP technology.