Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Images

TL;DR

This paper introduces a semi-supervised approach for 3D shape and pose estimation from 2D images, leveraging piecewise planar hull priors and consistency constraints to reduce annotation requirements while maintaining high accuracy.

Contribution

It proposes a novel semi-supervised framework that uses piecewise planar hull priors and consistency enforcement to improve 3D shape and pose estimation with limited labeled data.

Findings

Achieves comparable results to fully supervised methods with half the annotations.

Effectively leverages unlabeled data through planar hull consistency.

Introduces a new semi-supervised training framework for 3D shape and pose estimation.

Abstract

We study the problem of estimating 3D shape and pose of an object in terms of keypoints, from a single 2D image. The shape and pose are learned directly from images collected by categories and their partial 2D keypoint annotations.. In this work, we first propose an end-to-end training framework for intermediate 2D keypoints extraction and final 3D shape and pose estimation. The proposed framework is then trained using only the weak supervision of the intermediate 2D keypoints. Additionally, we devise a semi-supervised training framework that benefits from both labeled and unlabeled data. To leverage the unlabeled data, we introduce and exploit the \emph{piece-wise planar hull} prior of the canonical object shape. These planar hulls are defined manually once per object category, with the help of the keypoints. On the one hand, the proposed method learns to segment these planar hulls from the labeled data. On the other hand, it simultaneously enforces the consistency between predicted keypoints and the segmented hulls on the unlabeled data. The enforced consistency allows us to efficiently use the unlabeled data for the task at hand. The proposed method achieves comparable results with fully supervised state-of-the-art methods by using only half of the annotations. Our source code will be made publicly available.