A Probabilistic Graphical Model Approach to the Structure-and-Motion Problem

TL;DR

This paper introduces a probabilistic graphical model framework for the structure-and-motion problem in computer vision, utilizing Gaussian variables and belief propagation to estimate camera poses and 3D features from 2D images.

Contribution

It presents a novel probabilistic graphical model formulation with sigma point linearization and loopy belief propagation for structure-and-motion estimation.

Findings

Effective in simulation and real-world data

Flexible extension to additional parameters or constraints

Promising results in iterative estimation process

Abstract

We present a means of formulating and solving the well known structure-and-motion problem in computer vision with probabilistic graphical models. We model the unknown camera poses and 3D feature coordinates as well as the observed 2D projections as Gaussian random variables, using sigma point parameterizations to effectively linearize the nonlinear relationships between these variables. Those variables involved in every projection are grouped into a cluster, and we connect the clusters in a cluster graph. Loopy belief propagation is performed over this graph, in an iterative re-initialization and estimation procedure, and we find that our approach shows promise in both simulation and on real-world data. The PGM is easily extendable to include additional parameters or constraints.