Swipe Mosaics from Video

TL;DR

Swipe Mosaics is an interactive visualization method that places video frames on a 2D map representing scene layout, enabling easier capture and robust processing even in challenging conditions like dynamic scenes or low texture.

Contribution

The paper introduces a novel visual odometry algorithm that infers a distribution over camera motions, improving robustness over traditional deterministic methods.

Findings

Supports difficult scenes with low texture or dynamic content

Eases scene documentation and inspection

Robust to parallax and scene complexity

Abstract

A panoramic image mosaic is an attractive visualization for viewing many overlapping photos, but its images must be both captured and processed correctly to produce an acceptable composite. We propose Swipe Mosaics, an interactive visualization that places the individual video frames on a 2D planar map that represents the layout of the physical scene. Compared to traditional panoramic mosaics, our capture is easier because the user can both translate the camera center and film moving subjects. Processing and display degrade gracefully if the footage lacks distinct, overlapping, non-repeating texture. Our proposed visual odometry algorithm produces a distribution over (x,y) translations for image pairs. Inferring a distribution of possible camera motions allows us to better cope with parallax, lack of texture, dynamic scenes, and other phenomena that hurt deterministic reconstruction techniques. Robustness is obtained by training on synthetic scenes with known camera motions. We show that Swipe Mosaics are easy to generate, support a wide range of difficult scenes, and are useful for documenting a scene for closer inspection.