Test-time Correction: An Online 3D Detection System via Visual Prompting

TL;DR

This paper presents Test-time Correction (TTC), an online system that uses visual prompts and auxiliary feedback to correct errors in 3D detection during inference, enhancing autonomous driving safety without retraining.

Contribution

Introduction of TTC, a novel online 3D detection correction system utilizing visual prompts and auxiliary feedback for real-time error rectification during inference.

Findings

Significantly improves error correction over fixed 3D detectors

Operates effectively with limited labels and in zero-shot scenarios

Enhances detection reliability under adverse conditions

Abstract

This paper introduces Test-time Correction (TTC), an online 3D detection system designed to rectify test-time errors using various auxiliary feedback, aiming to enhance the safety of deployed autonomous driving systems. Unlike conventional offline 3D detectors that remain fixed during inference, TTC enables immediate online error correction without retraining, allowing autonomous vehicles to adapt to new scenarios and reduce deployment risks. To achieve this, we equip existing 3D detectors with an Online Adapter (OA) module -- a prompt-driven query generator for real-time correction. At the core of OA module are visual prompts: image-based descriptions of objects of interest derived from auxiliary feedback such as mismatches with 2D detections, road descriptions, or user clicks. These visual prompts, collected from risky objects during inference, are maintained in a visual prompt buffer to enable continuous correction in future frames. By leveraging this mechanism, TTC consistently detects risky objects, achieving reliable, adaptive, and versatile driving autonomy. Extensive experiments show that TTC significantly improves instant error rectification over frozen 3D detectors, even under limited labels, zero-shot settings, and adverse conditions. We hope this work inspires future research on post-deployment online rectification systems for autonomous driving.