EVE: Efficient zero-shot text-based Video Editing with Depth Map Guidance and Temporal Consistency Constraints

TL;DR

EVE is a zero-shot video editing method that uses depth maps and temporal constraints to achieve efficient and consistent edits, addressing the high cost and limited capacity issues of existing models.

Contribution

The paper introduces EVE, a novel zero-shot video editing approach leveraging depth guidance and temporal constraints, along with a new benchmark dataset ZVE-50.

Findings

EVE achieves a good balance between performance and efficiency.

EVE produces temporally consistent video edits.

The ZVE-50 dataset enables fair evaluation of video editing methods.

Abstract

Motivated by the superior performance of image diffusion models, more and more researchers strive to extend these models to the text-based video editing task. Nevertheless, current video editing tasks mainly suffer from the dilemma between the high fine-tuning cost and the limited generation capacity. Compared with images, we conjecture that videos necessitate more constraints to preserve the temporal consistency during editing. Towards this end, we propose EVE, a robust and efficient zero-shot video editing method. Under the guidance of depth maps and temporal consistency constraints, EVE derives satisfactory video editing results with an affordable computational and time cost. Moreover, recognizing the absence of a publicly available video editing dataset for fair comparisons, we construct a new benchmark ZVE-50 dataset. Through comprehensive experimentation, we validate that EVE could achieve a satisfactory trade-off between performance and efficiency. We will release our dataset and codebase to facilitate future researchers.