Physics-Aware Video Instance Removal Benchmark

TL;DR

The paper introduces PVIR, a benchmark for video instance removal that emphasizes physical realism, with annotated videos and evaluation of methods on complex physical interactions.

Contribution

It presents a new benchmark with annotated videos and a comprehensive evaluation protocol focusing on physical and semantic consistency in VIR.

Findings

PISCO-Removal and UniVideo achieve state-of-the-art results.

DiffuEraser often causes blurring artifacts.

Performance drops on the Hard subset highlight challenges in complex physical interactions.

Abstract

Video Instance Removal (VIR) requires removing target objects while maintaining background integrity and physical consistency, such as specular reflections and illumination interactions. Despite advancements in text-guided editing, current benchmarks primarily assess visual plausibility, often overlooking the physical causalities, such as lingering shadows, triggered by object removal. We introduce the Physics-Aware Video Instance Removal (PVIR) benchmark, featuring 95 high-quality videos annotated with instance-accurate masks and removal prompts. PVIR is partitioned into Simple and Hard subsets, the latter explicitly targeting complex physical interactions. We evaluate four representative methods, PISCO-Removal, UniVideo, DiffuEraser, and CoCoCo, using a decoupled human evaluation protocol across three dimensions to isolate semantic, visual, and spatial failures: instruction following, rendering quality, and edit exclusivity. Our results show that PISCO-Removal and UniVideo achieve state-of-the-art performance, while DiffuEraser frequently introduces blurring artifacts and CoCoCo struggles significantly with instruction following. The persistent performance drop on the Hard subset highlights the ongoing challenge of recovering complex physical side effects.