Resolving Long-Tail Ambiguity in Unsupervised 3D Point Cloud Segmentation with Language Priors

TL;DR

LangTail introduces a language-guided hierarchical learning framework that leverages language model priors to improve unsupervised 3D point cloud segmentation, especially for minority classes.

Contribution

It proposes a novel method combining language-derived semantic priors with hierarchical clustering to address long-tail ambiguity in 3D segmentation.

Findings

Outperforms existing methods with +13.5, +12.9, +8.9 mIoU improvements on ScanNet-v2, S3DIS, and nuScenes.

Effectively mitigates bias towards dominant classes in unsupervised 3D segmentation.

Demonstrates the benefit of language priors in enhancing minority class representation.

Abstract

Existing approaches for unsupervised 3D point cloud segmentation predominantly rely on a purely visual similarity-based learning-by-clustering paradigm, which suffers from a fundamental limitation: long-tail ambiguity. In such a paradigm, features of minor classes are consistently absorbed by dominant clusters, leading to severely imbalanced predictions. To address this issue, we propose LangTail, a language-guided hierarchical learning framework that leverages the balanced world knowledge encoded in language models to mitigate long-tail ambiguity in unsupervised 3D segmentation. The key idea is to establish multi-level associations between language-derived semantic priors and visually underrepresented minor classes, thereby compensating for the biased attention of purely visual clustering toward dominant classes. Specifically, LangTail first constructs an entity-level semantic prior from language models, capturing balanced and fine-grained world knowledge across categories. These priors are injected into a hierarchical clustering framework via contrastive alignment. This guides multi-granularity semantic structure formation and prevents minor classes from being absorbed by dominant clusters, yielding more discriminative representations for underrepresented categories. Extensive experiments on ScanNet-v2, S3DIS, and nuScenes demonstrate that LangTail consistently outperforms existing methods by significant margins, \ie, +13.5, +12.9, and +8.9 mIoU, respectively. These results demonstrate the effectiveness of language priors in improving the representation of minority classes in 3D point clouds. The code will be released at: https://github.com/Whisky0129/langtail_official.