Multilinguality in LLM-Designed Reward Functions for Restless Bandits: Effects on Task Performance and Fairness

TL;DR

This paper investigates how multilingual prompts influence the effectiveness and fairness of LLM-designed reward functions in Restless Multi-Armed Bandits, revealing language and complexity impacts on performance and bias.

Contribution

It is the first study to analyze multilingual effects on LLM-based reward functions in RMABs, highlighting language resource disparities and prompt complexity impacts.

Findings

English prompts yield better task performance than other languages.

Prompt phrasing significantly affects reward function quality.

Increased prompt complexity reduces performance and fairness, especially in low-resource languages.

Abstract

Restless Multi-Armed Bandits (RMABs) have been successfully applied to resource allocation problems in a variety of settings, including public health. With the rapid development of powerful large language models (LLMs), they are increasingly used to design reward functions to better match human preferences. Recent work has shown that LLMs can be used to tailor automated allocation decisions to community needs using language prompts. However, this has been studied primarily for English prompts and with a focus on task performance only. This can be an issue since grassroots workers, especially in developing countries like India, prefer to work in local languages, some of which are low-resource. Further, given the nature of the problem, biases along population groups unintended by the user are also undesirable. In this work, we study the effects on both task performance and fairness when the DLM algorithm, a recent work on using LLMs to design reward functions for RMABs, is prompted with non-English language commands. Specifically, we run the model on a synthetic environment for various prompts translated into multiple languages. The prompts themselves vary in complexity. Our results show that the LLM-proposed reward functions are significantly better when prompted in English compared to other languages. We also find that the exact phrasing of the prompt impacts task performance. Further, as prompt complexity increases, performance worsens for all languages; however, it is more robust with English prompts than with lower-resource languages. On the fairness side, we find that low-resource languages and more complex prompts are both highly likely to create unfairness along unintended dimensions.