PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models

TL;DR

This paper introduces PPU-Bench, a realistic benchmark for personalized partial unlearning in multimodal models, highlighting challenges and proposing Boundary-Aware Optimization to improve unlearning fidelity.

Contribution

The paper presents PPU-Bench, a new real-world benchmark for personalized unlearning in MLLMs, and proposes Boundary-Aware Optimization to better manage factual boundaries.

Findings

Complete unlearning often suppresses visual identity rather than factual knowledge.

Selective and personalized unlearning reveal significant forget-retain trade-offs.

Boundary-Aware Optimization improves enforcement of intra-subject factual boundaries.

Abstract

Multimodal Large Language Models (MLLMs) may memorize sensitive cross-modal information during pretraining. However, existing MLLM unlearning benchmarks rely on synthetic knowledge injection or complete subject-level deletion, which fail to capture realistic, personalized deletion requests that require fine-grained factual control. In this paper, we introduce PPU-Bench, a real-world and fine-tuning-free benchmark for personalized partial unlearning in MLLMs. PPU-Bench contains 24K multimodal and unimodal samples derived from pre-existing knowledge of 500 public figures under three progressively challenging settings: Complete, Selective, and Personalized unlearning. The benchmark evaluates whether methods can remove target knowledge while preserving non-target facts, model utility, and cross-modal consistency. Extensive experiments show that Complete Unlearning often suppresses visual identity rather than factual knowledge, while Selective and Personalized Unlearning expose significant forget--retain trade-offs and challenges in intra-subject factual boundaries. Robustness analysis under cross-image and prompt-based attacks reveals distinct vulnerabilities across different unlearning settings. Motivated by these findings, we propose Boundary-Aware Optimization (BAO), which explicitly models intra-subject forget-retain boundaries. Experimental results on two representative methods demonstrate that BAO can effectively enforce intra-subject factual boundaries.