FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing

TL;DR

FABLE introduces a hierarchical, fact-first framework for unstructured model editing that improves fine-grained fact access and question answering without sacrificing holistic editing performance.

Contribution

It proposes a novel two-stage, fact-anchoring approach and introduces UnFine, a benchmark for systematic evaluation of fine-grained fact editing in language models.

Findings

FABLE significantly enhances fine-grained question answering accuracy.

FABLE maintains state-of-the-art holistic editing performance.

UnFine provides a new benchmark with fact-level metrics for evaluation.

Abstract

Unstructured model editing aims to update models with real-world text, yet existing methods often memorize text holistically without reliable fine-grained fact access. To address this, we propose FABLE, a hierarchical framework that decouples fine-grained fact injection from holistic text generation. FABLE follows a two-stage, fact-first strategy: discrete facts are anchored in shallow layers, followed by minimal updates to deeper layers to produce coherent text. This decoupling resolves the mismatch between holistic recall and fine-grained fact access, reflecting the unidirectional Transformer flow in which surface-form generation amplifies rather than corrects underlying fact representations. We also introduce UnFine, a diagnostic benchmark with fine-grained question-answer pairs and fact-level metrics for systematic evaluation. Experiments show that FABLE substantially improves fine-grained question answering while maintaining state-of-the-art holistic editing performance. Our code is publicly available at https://github.com/caskcsg/FABLE.