Text3DAug -- Prompted Instance Augmentation for LiDAR Perception

TL;DR

Text3DAug introduces a novel, fully automated data augmentation method for LiDAR perception that generates instances and annotations from text, enhancing dataset diversity without manual effort or dependence on labeled data.

Contribution

It is the first approach to generate LiDAR instances and annotations directly from text, enabling automated, sensor-agnostic augmentation without manual curation.

Findings

Effective in LiDAR segmentation, detection, and class discovery

Performs on par or better than existing methods

Overcomes limitations of manual data annotation

Abstract

LiDAR data of urban scenarios poses unique challenges, such as heterogeneous characteristics and inherent class imbalance. Therefore, large-scale datasets are necessary to apply deep learning methods. Instance augmentation has emerged as an efficient method to increase dataset diversity. However, current methods require the time-consuming curation of 3D models or costly manual data annotation. To overcome these limitations, we propose Text3DAug, a novel approach leveraging generative models for instance augmentation. Text3DAug does not depend on labeled data and is the first of its kind to generate instances and annotations from text. This allows for a fully automated pipeline, eliminating the need for manual effort in practical applications. Additionally, Text3DAug is sensor agnostic and can be applied regardless of the LiDAR sensor used. Comprehensive experimental analysis on LiDAR segmentation, detection and novel class discovery demonstrates that Text3DAug is effective in supplementing existing methods or as a standalone method, performing on par or better than established methods, however while overcoming their specific drawbacks. The code is publicly available.