A Photogrammetry-based Framework to Facilitate Image-based Modeling and Automatic Camera Tracking

TL;DR

This paper introduces an open-source framework that integrates Structure from Motion and Multi-View Stereo techniques into Blender, enabling automatic camera tracking and dense scene modeling for image-based 3D reconstruction tasks.

Contribution

It extends Blender with a modular system supporting state-of-the-art SfM and MVS pipelines, automating camera motion estimation and dense scene reconstruction.

Findings

Automates camera motion estimation without manual feature tracking.

Enables dense scene modeling beyond Blender's built-in capabilities.

Supports multiple SfM and MVS pipelines for flexible workflows.

Abstract

We propose a framework that extends Blender to exploit Structure from Motion (SfM) and Multi-View Stereo (MVS) techniques for image-based modeling tasks such as sculpting or camera and motion tracking. Applying SfM allows us to determine camera motions without manually defining feature tracks or calibrating the cameras used to capture the image data. With MVS we are able to automatically compute dense scene models, which is not feasible with the built-in tools of Blender. Currently, our framework supports several state-of-the-art SfM and MVS pipelines. The modular system design enables us to integrate further approaches without additional effort. The framework is publicly available as an open source software package.