G2: Guided Generation for Enhanced Output Diversity in LLMs

TL;DR

G2 is a training-free method that improves output diversity in large language models by using guiding mechanisms during decoding, without sacrificing the quality of generated content.

Contribution

The paper introduces G2, a novel plug-and-play approach that enhances diversity in LLM outputs without additional training or quality loss.

Findings

G2 significantly increases output diversity in LLMs.

G2 maintains high output quality while improving diversity.

Experimental results validate G2's effectiveness across tasks.

Abstract

Large Language Models (LLMs) have demonstrated exceptional performance across diverse natural language processing tasks. However, these models exhibit a critical limitation in output diversity, often generating highly similar content across multiple attempts. This limitation significantly affects tasks requiring diverse outputs, from creative writing to reasoning. Existing solutions, like temperature scaling, enhance diversity by modifying probability distributions but compromise output quality. We propose Guide-to-Generation (G2), a training-free plug-and-play method that enhances output diversity while preserving generation quality. G2 employs a base generator alongside dual Guides, which guide the generation process through decoding-based interventions to encourage more diverse outputs conditioned on the original query. Comprehensive experiments demonstrate that G2 effectively improves output diversity while maintaining an optimal balance between diversity and quality.