KEditVis: A Visual Analytics System for Knowledge Editing of Large Language Models

TL;DR

KEditVis is a visual analytics tool that helps users understand and improve knowledge editing in large language models through interactive visualizations and targeted editing strategies.

Contribution

The paper introduces KEditVis, a novel system that enhances understanding and effectiveness of knowledge editing in LLMs with interactive visualizations.

Findings

KEditVis improves the identification of optimal editing layers.

Users can perform more targeted and effective knowledge edits.

Expert feedback confirms the system's usability and effectiveness.

Abstract

Large Language Models (LLMs) demonstrate exceptional capabilities in factual question answering, yet they sometimes provide incorrect responses. To address this issue, knowledge editing techniques have emerged as effective methods for correcting factual information in LLMs. However, typical knowledge editing workflows struggle with identifying the optimal set of model layers for editing and rely on summary indicators that provide insufficient guidance. This lack of transparency hinders effective comparison and identification of optimal editing strategies. In this paper, we present KEditVis, a novel visual analytics system designed to assist users in gaining a deeper understanding of knowledge editing through interactive visualizations, improving editing outcomes, and discovering valuable insights for the future development of knowledge editing algorithms. With KEditVis, users can select appropriate layers as the editing target, explore the reasons behind ineffective edits, and perform more targeted and effective edits. Our evaluation, including usage scenarios, expert interviews, and a user study, validates the effectiveness and usability of the system.