MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection

TL;DR

This paper introduces MOS, a test-time adaptation framework for LiDAR 3D object detection that dynamically combines historical model checkpoints to improve robustness against domain shifts and corruptions.

Contribution

The paper proposes a novel Model Synergy (MOS) strategy with Synergy Weights for effective online adaptation by assembling diverse historical checkpoints, addressing catastrophic forgetting and improving robustness.

Findings

Achieved 67.3% improvement in a cross-corruption scenario.

Demonstrated superior adaptability across three datasets and eight corruption types.

Effectively mitigated catastrophic forgetting through checkpoint selection and assembly.

Abstract

LiDAR-based 3D object detection is crucial for various applications but often experiences performance degradation in real-world deployments due to domain shifts. While most studies focus on cross-dataset shifts, such as changes in environments and object geometries, practical corruptions from sensor variations and weather conditions remain underexplored. In this work, we propose a novel online test-time adaptation framework for 3D detectors that effectively tackles these shifts, including a challenging cross-corruption scenario where cross-dataset shifts and corruptions co-occur. By leveraging long-term knowledge from previous test batches, our approach mitigates catastrophic forgetting and adapts effectively to diverse shifts. Specifically, we propose a Model Synergy (MOS) strategy that dynamically selects historical checkpoints with diverse knowledge and assembles them to best accommodate the current test batch. This assembly is directed by our proposed Synergy Weights (SW), which perform a weighted averaging of the selected checkpoints, minimizing redundancy in the composite model. The SWs are computed by evaluating the similarity of predicted bounding boxes on the test data and the independence of features between checkpoint pairs in the model bank. To maintain an efficient and informative model bank, we discard checkpoints with the lowest average SW scores, replacing them with newly updated models. Our method was rigorously tested against existing test-time adaptation strategies across three datasets and eight types of corruptions, demonstrating superior adaptability to dynamic scenes and conditions. Notably, it achieved a 67.3% improvement in a challenging cross-corruption scenario, offering a more comprehensive benchmark for adaptation. Source code: https://github.com/zhuoxiao-chen/MOS.