Zero-Shot Scene Change Detection

TL;DR

This paper introduces a training-free scene change detection method using tracking models that analyze content and style gaps between images, extending to videos for improved domain robustness and performance.

Contribution

The novel approach leverages tracking models for change detection without training, addressing content and style gaps, and extends to video analysis for enhanced robustness across domains.

Findings

Consistent performance across various domains.

Outperforms train-based baselines in diverse settings.

Effective extension to video for scene change detection.

Abstract

We present a novel, training-free approach to scene change detection. Our method leverages tracking models, which inherently perform change detection between consecutive frames of video by identifying common objects and detecting new or missing objects. Specifically, our method takes advantage of the change detection effect of the tracking model by inputting reference and query images instead of consecutive frames. Furthermore, we focus on the content gap and style gap between two input images in change detection, and address both issues by proposing adaptive content threshold and style bridging layers, respectively. Finally, we extend our approach to video, leveraging rich temporal information to enhance the performance of scene change detection. We compare our approach and baseline through various experiments. While existing train-based baseline tend to specialize only in the trained domain, our method shows consistent performance across various domains, proving the competitiveness of our approach.