TimeRewind: Rewinding Time with Image-and-Events Video Diffusion
Jingxi Chen, Brandon Y. Feng, Haoming Cai, Mingyang Xie, Christopher, Metzler, Cornelia Fermuller, Yiannis Aloimonos

TL;DR
TimeRewind introduces a novel method combining event camera data with diffusion models to generate plausible videos that rewind time from a single image, capturing missed moments with high temporal fidelity.
Contribution
The paper presents a new framework integrating neuromorphic event data with diffusion models to synthesize pre-capture videos from static images, addressing a previously unsolved problem.
Findings
Successfully generates high-quality rewind videos from single images.
Demonstrates the effectiveness of event-guided diffusion in capturing motion.
Opens new research directions in computational photography and generative modeling.
Abstract
This paper addresses the novel challenge of ``rewinding'' time from a single captured image to recover the fleeting moments missed just before the shutter button is pressed. This problem poses a significant challenge in computer vision and computational photography, as it requires predicting plausible pre-capture motion from a single static frame, an inherently ill-posed task due to the high degree of freedom in potential pixel movements. We overcome this challenge by leveraging the emerging technology of neuromorphic event cameras, which capture motion information with high temporal resolution, and integrating this data with advanced image-to-video diffusion models. Our proposed framework introduces an event motion adaptor conditioned on event camera data, guiding the diffusion model to generate videos that are visually coherent and physically grounded in the captured events. Through…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Video Analysis and Summarization · Cell Image Analysis Techniques
MethodsDiffusion
