Dense-SfM: Structure from Motion with Dense Consistent Matching

TL;DR

Dense-SfM introduces a dense, accurate 3D reconstruction framework from multi-view images by integrating dense matching, Gaussian Splatting, and transformer-based multi-view refinement, outperforming traditional sparse keypoint methods.

Contribution

It presents a novel SfM framework that combines dense matching, Gaussian Splatting, and transformer-based multi-view refinement for improved accuracy and density.

Findings

Significant accuracy improvements over state-of-the-art methods.

Enhanced point density in texture-less areas.

Robust multi-view track refinement using transformer and Gaussian Process.

Abstract

We present Dense-SfM, a novel Structure from Motion (SfM) framework designed for dense and accurate 3D reconstruction from multi-view images. Sparse keypoint matching, which traditional SfM methods often rely on, limits both accuracy and point density, especially in texture-less areas. Dense-SfM addresses this limitation by integrating dense matching with a Gaussian Splatting (GS) based track extension which gives more consistent, longer feature tracks. To further improve reconstruction accuracy, Dense-SfM is equipped with a multi-view kernelized matching module leveraging transformer and Gaussian Process architectures, for robust track refinement across multi-views. Evaluations on the ETH3D and Texture-Poor SfM datasets show that Dense-SfM offers significant improvements in accuracy and density over state-of-the-art methods. Project page: https://icetea-cv.github.io/densesfm/.