Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios

TL;DR

This paper introduces a novel environment-aware parameter generation method for object detection that adapts to changing scenarios, improving generalization without extensive fine-tuning.

Contribution

It proposes a dual-path LoRA-based adapter and a diffusion-based parameter generator for environment-conditioned adaptation in object detection.

Findings

Effective adaptation across diverse environments

Improved detection accuracy in dynamic scenarios

Enhanced generalization ability of the detector

Abstract

In practice, environments constantly change over time and space, posing significant challenges for object detectors trained based on a closed-set assumption, i.e., training and test data share the same distribution. To this end, continual test-time adaptation has attracted much attention, aiming to improve detectors' generalization by fine-tuning a few specific parameters, e.g., BatchNorm layers. However, based on a small number of test images, fine-tuning certain parameters may affect the representation ability of other fixed parameters, leading to performance degradation. Instead, we explore a new mechanism, i.e., converting the fine-tuning process to a specific-parameter generation. Particularly, we first design a dual-path LoRA-based domain-aware adapter that disentangles features into domain-invariant and domain-specific components, enabling efficient adaptation. Additionally, a conditional diffusion-based parameter generation mechanism is presented to synthesize the adapter's parameters based on the current environment, preventing the optimization from getting stuck in local optima. Finally, we propose a class-centered optimal transport alignment method to mitigate catastrophic forgetting. Extensive experiments conducted on various continuous domain adaptive object detection tasks demonstrate the effectiveness. Meanwhile, visualization results show that the representation extracted by the generated parameters can capture more object-related information and strengthen the generalization ability.