Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations

TL;DR

This paper introduces an unsupervised GAN-based approach for learning 3D geometry-aware representations of articulated objects without requiring pose annotations or masks, enabling controllable rendering.

Contribution

It presents a novel unsupervised method using GANs and tri-plane neural representations to learn controllable 3D models of articulated objects without paired supervision.

Findings

Efficient neural representation based on tri-planes reduces computational cost.

GAN training enables learning controllable 3D representations without supervision.

Method achieves realistic rendering of articulated objects with pose control.

Abstract

We propose an unsupervised method for 3D geometry-aware representation learning of articulated objects, in which no image-pose pairs or foreground masks are used for training. Though photorealistic images of articulated objects can be rendered with explicit pose control through existing 3D neural representations, these methods require ground truth 3D pose and foreground masks for training, which are expensive to obtain. We obviate this need by learning the representations with GAN training. The generator is trained to produce realistic images of articulated objects from random poses and latent vectors by adversarial training. To avoid a high computational cost for GAN training, we propose an efficient neural representation for articulated objects based on tri-planes and then present a GAN-based framework for its unsupervised training. Experiments demonstrate the efficiency of our method and show that GAN-based training enables the learning of controllable 3D representations without paired supervision.