TeFF: Tracking-enhanced Forgetting-free Few-shot 3D LiDAR Semantic Segmentation

TL;DR

This paper introduces TeFF, a method that leverages temporal continuity and a parameter-efficient training technique to improve few-shot 3D LiDAR semantic segmentation without forgetting previously learned classes.

Contribution

It proposes a tracking-based data augmentation approach combined with LoRA to address catastrophic forgetting in few-shot 3D LiDAR segmentation.

Findings

Enhanced segmentation accuracy on novel classes

Reduced forgetting of base classes

Effective data augmentation through tracking

Abstract

In autonomous driving, 3D LiDAR plays a crucial role in understanding the vehicle's surroundings. However, the newly emerged, unannotated objects presents few-shot learning problem for semantic segmentation. This paper addresses the limitations of current few-shot semantic segmentation by exploiting the temporal continuity of LiDAR data. Employing a tracking model to generate pseudo-ground-truths from a sequence of LiDAR frames, our method significantly augments the dataset, enhancing the model's ability to learn on novel classes. However, this approach introduces a data imbalance biased to novel data that presents a new challenge of catastrophic forgetting. To mitigate this, we incorporate LoRA, a technique that reduces the number of trainable parameters, thereby preserving the model's performance on base classes while improving its adaptability to novel classes. This work represents a significant step forward in few-shot 3D LiDAR semantic segmentation for autonomous driving. Our code is available at https://github.com/junbao-zhou/Track-no-forgetting.