DiffMVR: Diffusion-based Automated Multi-Guidance Video Restoration

TL;DR

DiffMVR is a diffusion-based video inpainting model that uses dynamic dual-guided prompts and adaptive references to accurately restore occluded regions in real-world, dynamic videos, especially for healthcare monitoring.

Contribution

The paper introduces DiffMVR, a novel diffusion-based video inpainting method with dynamic guidance and adaptive referencing, improving accuracy in complex, real-time scenarios.

Findings

Enhanced inpainting accuracy in dynamic environments

Effective handling of occlusions in real-world videos

Improved control over inpainting process

Abstract

In this work, we address a challenge in video inpainting: reconstructing occluded regions in dynamic, real-world scenarios. Motivated by the need for continuous human motion monitoring in healthcare settings, where facial features are frequently obscured, we propose a diffusion-based video-level inpainting model, DiffMVR. Our approach introduces a dynamic dual-guided image prompting system, leveraging adaptive reference frames to guide the inpainting process. This enables the model to capture both fine-grained details and smooth transitions between video frames, offering precise control over inpainting direction and significantly improving restoration accuracy in challenging, dynamic environments. DiffMVR represents a significant advancement in the field of diffusion-based inpainting, with practical implications for real-time applications in various dynamic settings.