Optimizing Diversity and Quality through Base-Aligned Model Collaboration

TL;DR

This paper introduces BACo, a novel inference-time model collaboration framework that dynamically combines aligned language models to enhance both diversity and quality of generated outputs, outperforming existing methods.

Contribution

We propose BACo, a new token-level model collaboration method that improves diversity and quality simultaneously at inference time, with strong controllability and state-of-the-art results.

Findings

BACo achieves a 21.3% joint improvement in diversity and quality.

It surpasses state-of-the-art inference-time baselines across multiple tasks and metrics.

Human evaluations confirm the effectiveness of BACo in real-world scenarios.

Abstract

Alignment has greatly improved large language models (LLMs)' output quality at the cost of diversity, yielding highly similar outputs across generations. We propose Base-Aligned Model Collaboration (BACo), an inference-time token-level model collaboration framework that dynamically combines a base LLM with its aligned counterpart to optimize diversity and quality. Inspired by prior work (Fei et al., 2025), BACo employs routing strategies that determine, at each token, from which model to decode based on next-token prediction uncertainty and predicted contents' semantic role. Prior diversity-promoting methods, such as retraining, prompt engineering, and multi-sampling methods, improve diversity but often degrade quality or require costly decoding or post-training. In contrast, BACo achieves both high diversity and quality post hoc within a single pass, while offering strong controllability. We explore a family of routing strategies, across three open-ended generation tasks and 13 metrics covering diversity and quality, BACo consistently surpasses state-of-the-art inference-time baselines. With our best router, BACo achieves a 21.3% joint improvement in diversity and quality. Human evaluations also mirror these improvements. The results suggest that collaboration between base and aligned models can optimize and control diversity and quality.