TimeReplayer: Unlocking the Potential of Event Cameras for Video Interpolation
Weihua He, Kaichao You, Zhendong Qiao, Xu Jia, Ziyang Zhang, Wenhui, Wang, Huchuan Lu, Yaoyuan Wang, Jianxing Liao

TL;DR
TimeReplayer leverages event camera data to enable high-quality video interpolation from low-FPS videos without high-speed training data, effectively modeling complex motions and allowing video extrapolation.
Contribution
The paper introduces a novel unsupervised cycle-consistent algorithm, TimeReplayer, that utilizes event camera data for video interpolation, eliminating the need for costly high-speed training data.
Findings
Achieves state-of-the-art video interpolation results.
Supports video extrapolation beyond original frames.
Demonstrates promising potential of event-based vision.
Abstract
Recording fast motion in a high FPS (frame-per-second) requires expensive high-speed cameras. As an alternative, interpolating low-FPS videos from commodity cameras has attracted significant attention. If only low-FPS videos are available, motion assumptions (linear or quadratic) are necessary to infer intermediate frames, which fail to model complex motions. Event camera, a new camera with pixels producing events of brightness change at the temporal resolution of second , is a game-changing device to enable video interpolation at the presence of arbitrarily complex motion. Since event camera is a novel sensor, its potential has not been fulfilled due to the lack of processing algorithms. The pioneering work Time Lens introduced event cameras to video interpolation by designing optical devices to collect a large amount of paired training data of high-speed frames…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Memory and Neural Computing · CCD and CMOS Imaging Sensors · Neural Networks and Reservoir Computing
