Symmetric Non-Rigid Structure from Motion for Category-Specific Object Structure Estimation

TL;DR

This paper develops symmetry-aware non-rigid structure from motion algorithms to improve 3D shape estimation of symmetric objects like cars and airplanes from multiple images, handling occlusions and non-rigid deformations.

Contribution

It extends existing non-rigid SfM methods by incorporating symmetry constraints, enabling better 3D reconstruction of symmetric objects from multiple views.

Findings

Significant performance improvements on Pascal3D+ dataset

Effective handling of occlusions and non-rigid deformations

Decoupling energy into symmetry-exploiting components

Abstract

Many objects, especially these made by humans, are symmetric, e.g. cars and aeroplanes. This paper addresses the estimation of 3D structures of symmetric objects from multiple images of the same object category, e.g. different cars, seen from various viewpoints. We assume that the deformation between different instances from the same object category is non-rigid and symmetric. In this paper, we extend two leading non-rigid structure from motion (SfM) algorithms to exploit symmetry constraints. We model the both methods as energy minimization, in which we also recover the missing observations caused by occlusions. In particularly, we show that by rotating the coordinate system, the energy can be decoupled into two independent terms, which still exploit symmetry, to apply matrix factorization separately on each of them for initialization. The results on the Pascal3D+ dataset show that our methods significantly improve performance over baseline methods.