MPF: Aligning and Debiasing Language Models post Deployment via Multi Perspective Fusion

TL;DR

MPF is a posttraining alignment framework that uses multiperspective generation to mitigate biases in large language models, aligning their outputs with humanlike baseline distributions without extensive finetuning.

Contribution

It introduces a novel multiperspective fusion approach for bias mitigation that is scalable, interpretable, and compatible with deployed LLMs, built on the SAGED pipeline.

Findings

Successfully aligns sentiment distributions with counterfactual and HR baselines

Reduces calibration error and KL divergence in LLM outputs

Generalizes well to unseen questions

Abstract

Multiperspective Fusion (MPF) is a novel posttraining alignment framework for large language models (LLMs) developed in response to the growing need for easy bias mitigation. Built on top of the SAGED pipeline, an automated system for constructing bias benchmarks and extracting interpretable baseline distributions, MPF leverages multiperspective generations to expose and align biases in LLM outputs with nuanced, humanlike baselines. By decomposing baseline, such as sentiment distributions from HR professionals, into interpretable perspective components, MPF guides generation through sampling and balancing of responses, weighted by the probabilities obtained in the decomposition. Empirically, we demonstrate its ability to align LLM sentiment distributions with both counterfactual baselines (absolute equality) and the HR baseline (biased for Top Univeristy), resulting in small KL divergence, reduction of calibration error and generalization to unseen questions. This shows that MPF offers a scalable and interpretable method for alignment and bias mitigation, compatible with deployed LLMs and requiring no extensive prompt engineering or finetuning.