HaineiFRDM: Explore Diffusion to Restore Defects in Fast-Movement Films

TL;DR

HaineiFRDM leverages diffusion models with innovative patch-wise training, frequency modules, and a global residual approach to significantly improve high-resolution film defect restoration, outperforming existing open-source methods.

Contribution

The paper introduces HaineiFRDM, a novel diffusion-based framework with new modules and strategies for high-resolution film restoration, including a unique dataset.

Findings

Outperforms existing open-source film restoration methods.

Effective high-resolution restoration with patch-wise training on limited GPU memory.

Demonstrates superior defect restoration capabilities through comprehensive experiments.

Abstract

Existing open-source film restoration methods show limited performance compared to commercial methods due to training with low-quality synthetic data and employing noisy optical flows. In addition, high-resolution films have not been explored by the open-source methods.We propose HaineiFRDM(Film Restoration Diffusion Model), a film restoration framework, to explore diffusion model's powerful content-understanding ability to help human expert better restore indistinguishable film defects.Specifically, we employ a patch-wise training and testing strategy to make restoring high-resolution films on one 24GB-VRAMR GPU possible and design a position-aware Global Prompt and Frame Fusion Modules.Also, we introduce a global-local frequency module to reconstruct consistent textures among different patches. Besides, we firstly restore a low-resolution result and use it as global residual to mitigate blocky artifacts caused by patching process.Furthermore, we construct a film restoration dataset that contains restored real-degraded films and realistic synthetic data.Comprehensive experimental results conclusively demonstrate the superiority of our model in defect restoration ability over existing open-source methods. Code and the dataset will be released.