AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction

TL;DR

AdaCo is a novel label-free learning approach that enhances 3D semantic segmentation by adaptively correcting noisy labels using cross-modal supervision, noise adjustment, and robust loss functions, outperforming existing methods on outdoor datasets.

Contribution

Introduces AdaCo, a new label-free learning framework with adaptive label correction, cross-modal supervision, and robust loss for improved 3D semantic segmentation in outdoor environments.

Findings

Outperforms existing methods on outdoor datasets

Effectively mitigates noise in supervision signals

Enhances 3D segmentation accuracy with adaptive techniques

Abstract

Recently, Visual Foundation Models (VFMs) have shown a remarkable generalization performance in 3D perception tasks. However, their effectiveness in large-scale outdoor datasets remains constrained by the scarcity of accurate supervision signals, the extensive noise caused by variable outdoor conditions, and the abundance of unknown objects. In this work, we propose a novel label-free learning method, Adaptive Label Correction (AdaCo), for 3D semantic segmentation. AdaCo first introduces the Cross-modal Label Generation Module (CLGM), providing cross-modal supervision with the formidable interpretive capabilities of the VFMs. Subsequently, AdaCo incorporates the Adaptive Noise Corrector (ANC), updating and adjusting the noisy samples within this supervision iteratively during training. Moreover, we develop an Adaptive Robust Loss (ARL) function to modulate each sample's sensitivity to noisy supervision, preventing potential underfitting issues associated with robust loss. Our proposed AdaCo can effectively mitigate the performance limitations of label-free learning networks in 3D semantic segmentation tasks. Extensive experiments on two outdoor benchmark datasets highlight the superior performance of our method.