Large Language Model Bias Mitigation from the Perspective of Knowledge Editing

TL;DR

This paper introduces a new benchmark and a fine-grained debiasing method for large language models that improves fairness without compromising knowledge accuracy.

Contribution

It proposes BiasKE for systematic bias assessment and FAST for editable fairness through knowledge calibration, advancing debiasing techniques in LLMs.

Findings

FAST outperforms existing debiasing methods

Maintains knowledge integrity while reducing bias

Provides a comprehensive bias mitigation benchmark

Abstract

Existing debiasing methods inevitably make unreasonable or undesired predictions as they are designated and evaluated to achieve parity across different social groups but leave aside individual facts, resulting in modified existing knowledge. In this paper, we first establish a new bias mitigation benchmark BiasKE leveraging existing and additional constructed datasets, which systematically assesses debiasing performance by complementary metrics on fairness, specificity, and generalization. Meanwhile, we propose a novel debiasing method, Fairness Stamp (FAST), which enables editable fairness through fine-grained calibration on individual biased knowledge. Comprehensive experiments demonstrate that FAST surpasses state-of-the-art baselines with remarkable debiasing performance while not hampering overall model capability for knowledge preservation, highlighting the prospect of fine-grained debiasing strategies for editable fairness in LLMs.