Learning to Detect 3D Reflection Symmetry for Single-View Reconstruction

TL;DR

This paper introduces a geometry-based deep learning framework that detects reflection symmetry planes in objects from a single image to improve 3D reconstruction accuracy, utilizing cost volumes for the first time in this context.

Contribution

It presents a novel end-to-end method combining symmetry detection and depth prediction, leveraging cost volumes to enhance single-view 3D reconstruction.

Findings

Outperforms previous state-of-the-art methods on ShapeNet dataset

Effectively utilizes symmetry cues for improved depth and pose estimation

Works even when objects are not fully symmetric

Abstract

3D reconstruction from a single RGB image is a challenging problem in computer vision. Previous methods are usually solely data-driven, which lead to inaccurate 3D shape recovery and limited generalization capability. In this work, we focus on object-level 3D reconstruction and present a geometry-based end-to-end deep learning framework that first detects the mirror plane of reflection symmetry that commonly exists in man-made objects and then predicts depth maps by finding the intra-image pixel-wise correspondence of the symmetry. Our method fully utilizes the geometric cues from symmetry during the test time by building plane-sweep cost volumes, a powerful tool that has been used in multi-view stereopsis. To our knowledge, this is the first work that uses the concept of cost volumes in the setting of single-image 3D reconstruction. We conduct extensive experiments on the ShapeNet dataset and find that our reconstruction method significantly outperforms the previous state-of-the-art single-view 3D reconstruction networks in term of the accuracy of camera poses and depth maps, without requiring objects being completely symmetric. Code is available at https://github.com/zhou13/symmetrynet.