Structure-Conditional Minimum Bayes Risk Decoding

TL;DR

This paper enhances Minimum Bayes Risk decoding by incorporating structural sensitivity, improving response quality in open-ended tasks like dialogue and instruction-following, demonstrated through curated datasets and real-world benchmarks.

Contribution

Introduces three lightweight utility function adaptations to make MBR more sensitive to structural variability in outcomes.

Findings

Structural utility adaptations improve MBR's structural optimality.

Enhanced MBR yields up to 13.7% higher win rate on benchmarks.

Common similarity-based utilities are insufficient for structural sensitivity.

Abstract

Minimum Bayes Risk (MBR) decoding has seen renewed interest as an alternative to traditional generation strategies. While MBR has proven effective in machine translation, where the variability of a language model's outcome space is naturally constrained, it may face challenges in more open-ended tasks such as dialogue or instruction-following. We hypothesise that in such settings, applying MBR with standard similarity-based utility functions may result in selecting responses that are broadly representative of the model's distribution, yet sub-optimal with respect to any particular grouping of generations that share an underlying latent structure. In this work, we introduce three lightweight adaptations to the utility function, designed to make MBR more sensitive to structural variability in the outcome space. To test our hypothesis, we curate a dataset capturing three representative types of latent structure: dialogue act, emotion, and response structure (e.g., a sentence, a paragraph, or a list). We further propose two metrics to evaluate the structural optimality of MBR. Our analysis demonstrates that common similarity-based utility functions fall short by these metrics. In contrast, our proposed adaptations considerably improve structural optimality. Finally, we evaluate our approaches on real-world instruction-following benchmarks, AlpacaEval and MT-Bench, and show that increased structural sensitivity improves generation quality by up to 13.7 percentage points in win rate.