Convolutional Networks for Object Category and 3D Pose Estimation from 2D Images

TL;DR

This paper presents a CNN architecture that jointly estimates object category and 3D pose from 2D images with known localization, achieving state-of-the-art results on PASCAL3D+.

Contribution

It introduces a novel CNN architecture with shared features and category-specific pose regressors for joint object categorization and pose estimation.

Findings

Achieves state-of-the-art performance on PASCAL3D+ for joint tasks.

Performance comparable to specialized pose estimation methods.

Proposes new loss functions and training strategies for the joint task.

Abstract

Current CNN-based algorithms for recovering the 3D pose of an object in an image assume knowledge about both the object category and its 2D localization in the image. In this paper, we relax one of these constraints and propose to solve the task of joint object category and 3D pose estimation from an image assuming known 2D localization. We design a new architecture for this task composed of a feature network that is shared between subtasks, an object categorization network built on top of the feature network, and a collection of category dependent pose regression networks. We also introduce suitable loss functions and a training method for the new architecture. Experiments on the challenging PASCAL3D+ dataset show state-of-the-art performance in the joint categorization and pose estimation task. Moreover, our performance on the joint task is comparable to the performance of state-of-the-art methods on the simpler 3D pose estimation with known object category task.