SOP^2: Transfer Learning with Scene-Oriented Prompt Pool on 3D Object Detection

TL;DR

This paper explores prompt tuning methods for 3D object detection, introducing a Scene-Oriented Prompt Pool (SOP^2) to enhance transfer learning from large datasets like Waymo.

Contribution

It proposes SOP^2, a novel prompt pool method, demonstrating its effectiveness in adapting 3D detection models across different scenarios.

Findings

Prompt pools improve 3D detection accuracy

Transfer learning from large datasets is effective

Scene-oriented prompts enhance model adaptability

Abstract

With the rise of Large Language Models (LLMs) such as GPT-3, these models exhibit strong generalization capabilities. Through transfer learning techniques such as fine-tuning and prompt tuning, they can be adapted to various downstream tasks with minimal parameter adjustments. This approach is particularly common in the field of Natural Language Processing (NLP). This paper aims to explore the effectiveness of common prompt tuning methods in 3D object detection. We investigate whether a model trained on the large-scale Waymo dataset can serve as a foundation model and adapt to other scenarios within the 3D object detection field. This paper sequentially examines the impact of prompt tokens and prompt generators, and further proposes a Scene-Oriented Prompt Pool (\textbf{SOP$^2$}). We demonstrate the effectiveness of prompt pools in 3D object detection, with the goal of inspiring future researchers to delve deeper into the potential of prompts in the 3D field.