mbrs: A Library for Minimum Bayes Risk Decoding

TL;DR

The paper introduces mbrs, an open-source library that implements Minimum Bayes Risk decoding for text generation, allowing flexible combination of metrics and efficient, transparent, reproducible decoding.

Contribution

It provides a flexible, efficient, and open-source implementation of MBR decoding, enhancing reproducibility and extensibility for research and development.

Findings

Supports various metrics and algorithms for MBR decoding

Focuses on speed, transparency, and reproducibility

Available as an MIT-licensed open-source library

Abstract

Minimum Bayes risk (MBR) decoding is a decision rule of text generation tasks that outperforms conventional maximum a posterior (MAP) decoding using beam search by selecting high-quality outputs based on a utility function rather than those with high-probability. Typically, it finds the most suitable hypothesis from the set of hypotheses under the sampled pseudo-references. mbrs is a library of MBR decoding, which can flexibly combine various metrics, alternative expectation estimations, and algorithmic variants. It is designed with a focus on speed measurement and calling count of code blocks, transparency, reproducibility, and extensibility, which are essential for researchers and developers. We published our mbrs as an MIT-licensed open-source project, and the code is available on GitHub. GitHub: https://github.com/naist-nlp/mbrs