CaseEdit: Enhancing Localized Commonsense Reasoning via Null-Space Constrained Knowledge Editing in Small Parameter Language Models

TL;DR

This paper introduces CaseEdit, a dataset and pipeline for evaluating personalized commonsense knowledge editing in small language models, demonstrating that null-space constrained editing improves knowledge integration with minimal interference.

Contribution

The paper presents CaseEdit, a novel dataset and method for localized knowledge editing in small LLMs, and introduces AlphaEdit, a null-space projection technique that enhances editing effectiveness.

Findings

AlphaEdit outperforms other editing methods on CaseEdit evaluations.

Null-space projection minimizes interference with unrelated knowledge.

Small models can effectively internalize personalized commonsense knowledge.

Abstract

Large language models (LLMs) exhibit strong performance on factual recall and general reasoning but struggle to adapt to user-specific, commonsense knowledge, a challenge particularly acute in small-parameter settings where computational efficiency is prioritized. We introduce CaseEdit, a new dataset and generation pipeline for evaluating localized, personalized commonsense knowledge editing in small LLMs to address this. Built upon the ATOMIC20/20 commonsense graph, CaseEdit uses a multi-stage inference process to generate both typical and atypical contextual edits for household objects, paired with targeted evaluation questions across four axes: reliability, generalization, locality, and portability. We evaluate established knowledge editing methods using CaseEdit and demonstrate that AlphaEdit, a technique employing null-space projection to minimize interference with unrelated knowledge, consistently outperforms other methods when applied to an LLaMA 3.2 3B model, even in scalability tests, showing minimal ripple effects. Our results indicate that using CaseEdit with effective editing techniques like AlphaEdit allows small models to internalize high-quality, context-sensitive common-sense knowledge, paving the way for lightweight, personalized assistants.