Beyond Rigid: Benchmarking Non-Rigid Video Editing

TL;DR

This paper introduces NRVBench, a comprehensive benchmark for evaluating non-rigid video editing, including a new dataset, evaluation metric, and a training-free baseline, to improve physical plausibility and temporal consistency.

Contribution

The paper presents the first dedicated benchmark for non-rigid video editing, including a dataset, a novel evaluation metric, and a structure-aware editing baseline.

Findings

Current methods struggle with physical plausibility.

The proposed VM-Edit baseline outperforms existing approaches.

NRVBench provides a standard platform for physics-aware video editing evaluation.

Abstract

Despite the remarkable progress in text-driven video editing, generating coherent non-rigid deformations remains a critical challenge, often plagued by physical distortion and temporal flicker. To bridge this gap, we propose NRVBench, the first dedicated and comprehensive benchmark designed to evaluate non-rigid video editing. First, we curate a high-quality dataset consisting of 180 non-rigid motion videos from six physics-based categories, equipped with 2,340 fine-grained task instructions and 360 multiple-choice questions. Second, we propose NRVE-Acc, a novel evaluation metric based on Vision-Language Models that can rigorously assess physical compliance, temporal consistency, and instruction alignment, overcoming the limitations of general metrics in capturing complex dynamics. Third, we introduce a training-free baseline, VM-Edit, which utilizes a dual-region denoising mechanism to achieve structure-aware control, balancing structural preservation and dynamic deformation. Extensive experiments demonstrate that while current methods have shortcomings in maintaining physical plausibility, our method achieves excellent performance across both standard and proposed metrics. We believe the benchmark could serve as a standard testing platform for advancing physics-aware video editing.