3D Compositional Zero-shot Learning with DeCompositional Consensus

TL;DR

This paper introduces a new approach for 3D compositional zero-shot learning in semantic segmentation, leveraging part generalization and a novel DeCompositional Consensus method to improve generalization to unseen object classes.

Contribution

It proposes DeCompositional Consensus, a novel method combining segmentation and scoring networks, and provides a structured benchmark with the Compositional-PartNet dataset for the task.

Findings

DeCompositional Consensus outperforms existing methods in zero-shot segmentation.

The Compositional-PartNet dataset enables effective benchmarking of part-based generalization.

The approach achieves state-of-the-art results in both zero-shot segmentation and classification.

Abstract

Parts represent a basic unit of geometric and semantic similarity across different objects. We argue that part knowledge should be composable beyond the observed object classes. Towards this, we present 3D Compositional Zero-shot Learning as a problem of part generalization from seen to unseen object classes for semantic segmentation. We provide a structured study through benchmarking the task with the proposed Compositional-PartNet dataset. This dataset is created by processing the original PartNet to maximize part overlap across different objects. The existing point cloud part segmentation methods fail to generalize to unseen object classes in this setting. As a solution, we propose DeCompositional Consensus, which combines a part segmentation network with a part scoring network. The key intuition to our approach is that a segmentation mask over some parts should have a consensus with its part scores when each part is taken apart. The two networks reason over different part combinations defined in a per-object part prior to generate the most suitable segmentation mask. We demonstrate that our method allows compositional zero-shot segmentation and generalized zero-shot classification, and establishes the state of the art on both tasks.