Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a Single-View Image

TL;DR

This paper presents an unsupervised, template-based method for reconstructing severely deformed 3D meshes from a single-view image, enabling accurate downstream length measurement without 3D ground truth.

Contribution

It introduces a novel unsupervised approach that reconstructs 3D meshes of severely deformed objects from a single image, suitable for downstream tasks like length measurement.

Findings

Achieves state-of-the-art accuracy in length measurement on a severely deformed fish dataset.

Reconstructs 3D meshes without using 3D ground truth.

Effectively handles severely deformed objects in a single-view setting.

Abstract

Much progress has been made in the supervised learning of 3D reconstruction of rigid objects from multi-view images or a video. However, it is more challenging to reconstruct severely deformed objects from a single-view RGB image in an unsupervised manner. Although training-based methods, such as specific category-level training, have been shown to successfully reconstruct rigid objects and slightly deformed objects like birds from a single-view image, they cannot effectively handle severely deformed objects and neither can be applied to some downstream tasks in the real world due to the inconsistent semantic meaning of vertices, which are crucial in defining the adopted 3D templates of objects to be reconstructed. In this work, we introduce a template-based method to infer 3D shapes from a single-view image and apply the reconstructed mesh to a downstream task, i.e., absolute length measurement. Without using 3D ground truth, our method faithfully reconstructs 3D meshes and achieves state-of-the-art accuracy in a length measurement task on a severely deformed fish dataset.