3D Pose Estimation for Fine-Grained Object Categories

TL;DR

This paper introduces a new dataset and framework for fine-grained object pose estimation, leveraging dense 3D models and CNNs to improve accuracy beyond previous category-level methods.

Contribution

The work provides a large, annotated dataset for fine-grained 3D pose estimation and proposes a CNN-based framework utilizing dense 3D representations for enhanced accuracy.

Findings

Full perspective pose estimation is achievable with 2D appearance data.

Dense 3D representations improve pose estimation performance.

The dataset enables benchmarking for fine-grained object pose estimation.

Abstract

Existing object pose estimation datasets are related to generic object types and there is so far no dataset for fine-grained object categories. In this work, we introduce a new large dataset to benchmark pose estimation for fine-grained objects, thanks to the availability of both 2D and 3D fine-grained data recently. Specifically, we augment two popular fine-grained recognition datasets (StanfordCars and CompCars) by finding a fine-grained 3D CAD model for each sub-category and manually annotating each object in images with 3D pose. We show that, with enough training data, a full perspective model with continuous parameters can be estimated using 2D appearance information alone. We achieve this via a framework based on Faster/Mask R-CNN. This goes beyond previous works on category-level pose estimation, which only estimate discrete/continuous viewpoint angles or recover rotation matrices often with the help of key points. Furthermore, with fine-grained 3D models available, we incorporate a dense 3D representation named as location field into the CNN-based pose estimation framework to further improve the performance. The new dataset is available at www.umiacs.umd.edu/~wym/3dpose.html