DaBiT: Depth and Blur informed Transformer for Video Focal Deblurring

TL;DR

This paper introduces DaBiT, a novel transformer-based framework for video focal deblurring that leverages map-guided transformers, flow re-focusing, and synthetic data generation to achieve state-of-the-art results.

Contribution

The paper presents a new approach for video focal deblurring using map-guided transformers and introduces a synthetic dataset, DAVIS-Blur, for training and benchmarking.

Findings

Achieves 1.9dB higher PSNR than existing methods

Introduces a new dataset DAVIS-Blur with realistic focal blur

Demonstrates superior performance through comprehensive experiments

Abstract

In many real-world scenarios, recorded videos suffer from accidental focus blur, and while video deblurring methods exist, most specifically target motion blur or spatial-invariant blur. This paper introduces a framework optimized for the as yet unattempted task of video focal deblurring (refocusing). The proposed method employs novel map-guided transformers, in addition to image propagation, to effectively leverage the continuous spatial variance of focal blur and restore the footage. We also introduce a flow re-focusing module designed to efficiently align relevant features between blurry and sharp domains. Additionally, we propose a novel technique for generating synthetic focal blur data, broadening the model's learning capabilities and robustness to include a wider array of content. We have made a new benchmark dataset, DAVIS-Blur, available. This dataset, a modified extension of the popular DAVIS video segmentation set, provides realistic focal blur degradations as well as the corresponding blur maps. Comprehensive experiments demonstrate the superiority of our approach. We achieve state-of-the-art results with an average PSNR performance over 1.9dB greater than comparable existing video restoration methods. Our source code and the developed databases will be made available at https://github.com/crispianm/DaBiT