Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects

TL;DR

This paper introduces an unsupervised method to discover 3D joints and structure of unseen articulated objects from multiple views, enabling pose manipulation without prior structural annotations.

Contribution

It presents a novel approach that learns both appearance and structure of new articulated objects using only movement observations, without any joint supervision.

Findings

Works across various object types, including animals and robots.

Accurately identifies joints and parts without prior structural knowledge.

Effective in re-posing unseen articulated objects.

Abstract

Rendering articulated objects while controlling their poses is critical to applications such as virtual reality or animation for movies. Manipulating the pose of an object, however, requires the understanding of its underlying structure, that is, its joints and how they interact with each other. Unfortunately, assuming the structure to be known, as existing methods do, precludes the ability to work on new object categories. We propose to learn both the appearance and the structure of previously unseen articulated objects by observing them move from multiple views, with no joints annotation supervision, or information about the structure. We observe that 3D points that are static relative to one another should belong to the same part, and that adjacent parts that move relative to each other must be connected by a joint. To leverage this insight, we model the object parts in 3D as ellipsoids, which allows us to identify joints. We combine this explicit representation with an implicit one that compensates for the approximation introduced. We show that our method works for different structures, from quadrupeds, to single-arm robots, to humans.