Learning Robust Video Synchronization without Annotations

TL;DR

This paper introduces a scalable, unsupervised method for robust non-linear temporal video alignment that autonomously learns from data without manual labels, effectively handling videos recorded months apart.

Contribution

It presents a novel unsupervised, iterative learning approach for video synchronization that does not require manual annotations and can handle long-term variations.

Findings

Successfully aligns videos recorded months apart

Operates without manual annotations or labels

Demonstrates robustness to seasonal and weather changes

Abstract

Aligning video sequences is a fundamental yet still unsolved component for a broad range of applications in computer graphics and vision. Most classical image processing methods cannot be directly applied to related video problems due to the high amount of underlying data and their limit to small changes in appearance. We present a scalable and robust method for computing a non-linear temporal video alignment. The approach autonomously manages its training data for learning a meaningful representation in an iterative procedure each time increasing its own knowledge. It leverages on the nature of the videos themselves to remove the need for manually created labels. While previous alignment methods similarly consider weather conditions, season and illumination, our approach is able to align videos from data recorded months apart.