EgoSampling: Wide View Hyperlapse from Egocentric Videos

TL;DR

EgoSampling introduces an adaptive frame sampling method for egocentric videos that produces stable, wide-view hyperlapse videos by minimizing shake and mosaicing frames, even from multiple videos, for faster viewing.

Contribution

The paper presents a novel energy minimization approach for adaptive frame sampling that stabilizes and widens egocentric hyperlapse videos, including from multiple sources.

Findings

Produces stable, fast-forwarded hyperlapse videos with increased field-of-view.

Frames are mosaiced to enhance view and stability.

Method enables combining multiple videos into a single hyperlapse.

Abstract

The possibility of sharing one's point of view makes use of wearable cameras compelling. These videos are often long, boring and coupled with extreme shake, as the camera is worn on a moving person. Fast forwarding (i.e. frame sampling) is a natural choice for quick video browsing. However, this accentuates the shake caused by natural head motion in an egocentric video, making the fast forwarded video useless. We propose EgoSampling, an adaptive frame sampling that gives stable, fast forwarded, hyperlapse videos. Adaptive frame sampling is formulated as an energy minimization problem, whose optimal solution can be found in polynomial time. We further turn the camera shake from a drawback into a feature, enabling the increase in field-of-view of the output video. This is obtained when each output frame is mosaiced from several input frames. The proposed technique also enables the generation of a single hyperlapse video from multiple egocentric videos, allowing even faster video consumption.