Unblur-SLAM: Dense Neural SLAM for Blurry Inputs

TL;DR

Unblur-SLAM introduces a neural SLAM pipeline capable of producing sharp 3D reconstructions from blurred images by handling various blur types and integrating deblurring with pose and mapping refinement.

Contribution

It presents a novel approach combining feed-forward deblurring, multi-view optimization, and a 3D blur model to improve SLAM performance on blurry inputs.

Findings

Achieves state-of-the-art results in handling motion and defocus blur.

Improves pose estimation accuracy and 3D reconstruction sharpness.

Effectively models blur formation in 3D space for better detail recovery.

Abstract

We propose Unblur-SLAM, a novel RGB SLAM pipeline for sharp 3D reconstruction from blurred image inputs. In contrast to previous work, our approach is able to handle different types of blur and demonstrates state-of-the-art performance in the presence of both motion blur and defocus blur. Moreover, we adjust the computation effort with the amount of blur in the input image. As a first stage, our method uses a feed-forward image deblurring model for which we propose a suitable training scheme that can improve both tracking and mapping modules. Frames that are successfully deblurred by the feed-forward network obtain refined poses and depth through local-global multi-view optimization and loop closure. Frames that fail the first stage deblurring are directly modeled through the global 3DGS representation and an additional blur network to model multiple blurred sub-frames and simulate the blur formation process in 3D space, thereby learning sharp details and refined sub-frame poses. Experiments on several real-world datasets demonstrate consistent improvements in both pose estimation and sharp reconstruction results of geometry and texture.