Top-Down Beats Bottom-Up in 3D Instance Segmentation

TL;DR

TD3D introduces a novel top-down, cluster-free, fully-convolutional approach for 3D instance segmentation that surpasses bottom-up methods in accuracy and speed across multiple benchmarks.

Contribution

It presents the first top-down, end-to-end trainable 3D instance segmentation method that outperforms existing bottom-up approaches in accuracy and efficiency.

Findings

Outperforms bottom-up methods on standard benchmarks

Achieves 1.9x faster inference than state-of-the-art bottom-up methods

Demonstrates superior accuracy and generalization across datasets

Abstract

Most 3D instance segmentation methods exploit a bottom-up strategy, typically including resource-exhaustive post-processing. For point grouping, bottom-up methods rely on prior assumptions about the objects in the form of hyperparameters, which are domain-specific and need to be carefully tuned. On the contrary, we address 3D instance segmentation with a TD3D: the pioneering cluster-free, fully-convolutional and entirely data-driven approach trained in an end-to-end manner. This is the first top-down method outperforming bottom-up approaches in 3D domain. With its straightforward pipeline, it demonstrates outstanding accuracy and generalization ability on the standard indoor benchmarks: ScanNet v2, its extension ScanNet200, and S3DIS, as well as on the aerial STPLS3D dataset. Besides, our method is much faster on inference than the current state-of-the-art grouping-based approaches: our flagship modification is 1.9x faster than the most accurate bottom-up method, while being more accurate, and our faster modification shows state-of-the-art accuracy running at 2.6x speed. Code is available at https://github.com/SamsungLabs/td3d .