PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understanding

TL;DR

PointTPA introduces a test-time parameter adaptation framework for 3D scene understanding that dynamically generates input-aware network parameters, significantly improving performance with minimal additional parameters.

Contribution

It proposes a novel dynamic parameter adaptation method using SNG and DPP modules, enabling scene-specific network adjustments during inference.

Findings

Achieves 78.4% mIoU on ScanNet validation.

Surpasses existing PEFT methods across benchmarks.

Introduces lightweight modules with less than 2% of backbone parameters.

Abstract

Scene-level point cloud understanding remains challenging due to diverse geometries, imbalanced category distributions, and highly varied spatial layouts. Existing methods improve object-level performance but rely on static network parameters during inference, limiting their adaptability to dynamic scene data. We propose PointTPA, a Test-time Parameter Adaptation framework that generates input-aware network parameters for scene-level point clouds. PointTPA adopts a Serialization-based Neighborhood Grouping (SNG) to form locally coherent patches and a Dynamic Parameter Projector (DPP) to produce patch-wise adaptive weights, enabling the backbone to adjust its behavior according to scene-specific variations while maintaining a low parameter overhead. Integrated into the PTv3 structure, PointTPA demonstrates strong parameter efficiency by introducing two lightweight modules of less than 2% of the backbone's parameters. Despite this minimal parameter overhead, PointTPA achieves 78.4% mIoU on ScanNet validation, surpassing existing parameter-efficient fine-tuning (PEFT) methods across multiple benchmarks, highlighting the efficacy of our test-time dynamic network parameter adaptation mechanism in enhancing 3D scene understanding. The code is available at https://github.com/H-EmbodVis/PointTPA.