CVAM-Pose: Conditional Variational Autoencoder for Multi-Object Monocular Pose Estimation

TL;DR

CVAM-Pose introduces a novel label-embedded variational autoencoder for multi-object monocular pose estimation, achieving high accuracy without reliance on 3D models or depth data, and demonstrating robustness to occlusion and clutter.

Contribution

The paper presents a new approach using a label-embedded conditional variational autoencoder for scalable, efficient multi-object pose estimation from monocular images, outperforming existing latent space methods.

Findings

Outperforms AAE and Multi-Path methods by 20-25% on AR_VSD metric.

Robust to occlusion and clutter in multi-object scenes.

Comparable to 3D model-based methods in BOP challenges.

Abstract

Estimating rigid objects' poses is one of the fundamental problems in computer vision, with a range of applications across automation and augmented reality. Most existing approaches adopt one network per object class strategy, depend heavily on objects' 3D models, depth data, and employ a time-consuming iterative refinement, which could be impractical for some applications. This paper presents a novel approach, CVAM-Pose, for multi-object monocular pose estimation that addresses these limitations. The CVAM-Pose method employs a label-embedded conditional variational autoencoder network, to implicitly abstract regularised representations of multiple objects in a single low-dimensional latent space. This autoencoding process uses only images captured by a projective camera and is robust to objects' occlusion and scene clutter. The classes of objects are one-hot encoded and embedded throughout the network. The proposed label-embedded pose regression strategy interprets the learnt latent space representations utilising continuous pose representations. Ablation tests and systematic evaluations demonstrate the scalability and efficiency of the CVAM-Pose method for multi-object scenarios. The proposed CVAM-Pose outperforms competing latent space approaches. For example, it is respectively 25% and 20% better than AAE and Multi-Path methods, when evaluated using the $\mathrm{AR_{VSD}}$ metric on the Linemod-Occluded dataset. It also achieves results somewhat comparable to methods reliant on 3D models reported in BOP challenges. Code available: https://github.com/JZhao12/CVAM-Pose