Canonical Space Representation for 4D Panoptic Segmentation of Articulated Objects

TL;DR

This paper introduces a new dataset and a novel 4D panoptic segmentation framework for articulated objects, leveraging temporal dynamics and canonical space alignment to improve segmentation accuracy in dynamic scenarios.

Contribution

The paper presents Artic4D, a new dataset with 4D panoptic annotations, and CanonSeg4D, a framework that explicitly models temporal dynamics using canonical space for improved segmentation.

Findings

CanonSeg4D outperforms existing methods in complex scenarios

Temporal modeling improves segmentation consistency

Canonical alignment enhances part-level segmentation accuracy

Abstract

Articulated object perception presents significant challenges in computer vision, particularly because most existing methods ignore temporal dynamics despite the inherently dynamic nature of such objects. The use of 4D temporal data has not been thoroughly explored in articulated object perception and remains unexamined for panoptic segmentation. The lack of a benchmark dataset further hurt this field. To this end, we introduce Artic4D as a new dataset derived from PartNet Mobility and augmented with synthetic sensor data, featuring 4D panoptic annotations and articulation parameters. Building on this dataset, we propose CanonSeg4D, a novel 4D panoptic segmentation framework. This approach explicitly estimates per-frame offsets mapping observed object parts to a learned canonical space, thereby enhancing part-level segmentation. The framework employs this canonical representation to achieve consistent alignment of object parts across sequential frames. Comprehensive experiments on Artic4D demonstrate that the proposed CanonSeg4D outperforms state of the art approaches in panoptic segmentation accuracy in more complex scenarios. These findings highlight the effectiveness of temporal modeling and canonical alignment in dynamic object understanding, and pave the way for future advances in 4D articulated object perception.