Addressing LLM Diversity by Infusing Random Concepts

TL;DR

This paper demonstrates that infusing random concepts into prompts can significantly enhance the diversity of outputs generated by large language models, using a new evaluation protocol for systematic benchmarking.

Contribution

It introduces a novel method of infusing randomness into prompts to improve LLM output diversity and proposes an evaluation protocol for benchmarking this diversity.

Findings

Infusing random words increases output diversity across multiple LLMs.

Prepending unrelated sentences leads to more varied responses.

The evaluation protocol can be used for systematic diversity benchmarking.

Abstract

Large language models (LLMs) are known to produce outputs with limited diversity. In this work, we study whether infusing random concepts in the prompts can improve the diversity of the generated outputs. To benchmark the approach, we design a systematic evaluation protocol which involves prompting an LLM with questions of the form "Name 10 Hollywood actors", and analyzing diversity measures of the resulting LLM outputs. Our experiments on multiple LLMs show that prepending random words/sentences unrelated to the prompt result in greater diversity in the outputs of LLMs. We believe that this promising result and the evaluation protocol opens up interesting avenues for future work, such as how infusing randomness into LLMs could be applied to other domains. Further, the evaluation protocol could also inspire research into benchmarking LLM diversity more systematically.