Improved Convex Decomposition with Ensembling and Negative Primitives

TL;DR

This paper introduces a novel scene representation method using positive and negative primitives with ensembling to improve geometric fitting and segmentation accuracy, validated on NYUv2 and LAION datasets.

Contribution

It presents a new approach that incorporates negative primitives and ensembling to enhance convex decomposition and scene modeling.

Findings

Significant improvements in depth and segmentation accuracy over state-of-the-art methods.

Negative primitives notably improve fitting precision.

Method is robust across different datasets, including NYUv2 and LAION.

Abstract

Describing a scene in terms of primitives -- geometrically simple shapes that offer a parsimonious but accurate abstraction of structure -- is an established and difficult fitting problem. Different scenes require different numbers of primitives, and these primitives interact strongly. Existing methods are evaluated by comparing predicted depth, normals, and segmentation against ground truth. The state of the art method involves a learned regression procedure to predict a start point consisting of a fixed number of primitives, followed by a descent method to refine the geometry and remove redundant primitives. CSG (Constructive Solid Geometry) representations are significantly enhanced by a set-differencing operation. Our representation incorporates negative primitives, which are differenced from the positive primitives. These notably enrich the geometry that the model can encode, while complicating the fitting problem. This paper presents a method that can (a) incorporate these negative primitives and (b) choose the overall number of positive and negative primitives by ensembling. Extensive experiments on the standard NYUv2 dataset confirm that (a) this approach results in substantial improvements in depth representation and segmentation over SOTA and (b) negative primitives improve fitting accuracy. Our method is robustly applicable across datasets: in a first, we evaluate primitive prediction for LAION images.