Video Reconstruction using Diffusion-based Image-to-Video Generation with Trajectory Guidance

TL;DR

This paper introduces a diffusion-based method for reconstructing missing drone video frames using GPS trajectory guidance, achieving realistic and trajectory-adherent results without domain-specific fine-tuning.

Contribution

It presents a novel pipeline that converts GPS telemetry and a reference frame into a trajectory-guided video sequence using a pre-trained diffusion model, eliminating the need for fine-tuning.

Findings

SG-I2V produces more natural frames than baselines.

The method achieves better motion realism and trajectory adherence.

It outperforms optical flow and RIFE interpolation in key metrics.

Abstract

This paper addresses the problem of reconstructing missing or dropped frames in top-down drone video of autonomous surface vehicles performing structured maritime manoeuvres. We propose a pipeline that converts raw GPS telemetry and a single reference frame into a trajectory-guided video sequence using a pre-trained image-to-video diffusion model, requiring no domain-specific fine-tuning. GPS coordinates from onboard telemetry logs are projected into image space via an equirectangular mapping, producing per-vessel motion cues that condition the SG-I2V diffusion model. The generated frames are evaluated against ground-truth video using perceptual, temporal and trajectory-based metrics, and benchmarked against optical flow extrapolation and RIFE interpolation baselines. SG-I2V produces the most naturally appearing frames among all methods (BRISQUE 25.52, closest to ground-truth 23.64), the most realistic motion magnitude (temporal smoothness 1.14 vs. ground truth 1.42), and the strongest GPS trajectory adherence (9.31px vs. 28.70px for ground-truth, the latter reflecting approximate temporal alignment between footage and GPS logs rather than generation error), demonstrating that trajectory-guided diffusion synthesis is a viable approach to maritime video reconstruction under challenging low-texture, small-object conditions.