LiveMoments: Reselected Key Photo Restoration in Live Photos via Reference-guided Diffusion

TL;DR

LiveMoments is a neural network framework that restores reselected key photos in Live Photos by leveraging reference images and motion guidance, significantly improving quality in challenging scenes.

Contribution

It introduces a reference-guided restoration method with a motion alignment module specifically for reselected Live Photo key images.

Findings

Significantly improves perceptual quality of reselected key photos.

Effective in scenes with fast motion or complex structures.

Outperforms existing solutions in experiments.

Abstract

Live Photo captures both a high-quality key photo and a short video clip to preserve the precious dynamics around the captured moment. While users may choose alternative frames as the key photo to capture better expressions or timing, these frames often exhibit noticeable quality degradation, as the photo capture ISP pipeline delivers significantly higher image quality than the video pipeline. This quality gap highlights the need for dedicated restoration techniques to enhance the reselected key photo. To this end, we propose LiveMoments, a reference-guided image restoration framework tailored for the reselected key photo in Live Photos. Our method employs a two-branch neural network: a reference branch that extracts structural and textural information from the original high-quality key photo, and a main branch that restores the reselected frame using the guidance provided by the reference branch. Furthermore, we introduce a unified Motion Alignment module that incorporates motion guidance for spatial alignment at both the latent and image levels. Experiments on real and synthetic Live Photos demonstrate that LiveMoments significantly improves perceptual quality and fidelity over existing solutions, especially in scenes with fast motion or complex structures. Our code is available at https://github.com/OpenVeraTeam/LiveMoments.