Online Learning via Memory: Retrieval-Augmented Detector Adaptation

TL;DR

This paper introduces a training-free online domain adaptation method for object detectors using a retrieval-augmented memory module, enabling quick adaptation to new domains by leveraging stored knowledge during testing.

Contribution

It proposes a novel retrieval-augmented classification approach with a memory bank for real-time detector adaptation without retraining.

Findings

Significantly outperforms baselines in domain adaptation tasks.

Effective with minimal memory (e.g., 10 images per category).

Works with both open-set and close-set detectors.

Abstract

This paper presents a novel way of online adapting any off-the-shelf object detection model to a novel domain without retraining the detector model. Inspired by how humans quickly learn knowledge of a new subject (e.g., memorization), we allow the detector to look up similar object concepts from memory during test time. This is achieved through a retrieval augmented classification (RAC) module together with a memory bank that can be flexibly updated with new domain knowledge. We experimented with various off-the-shelf open-set detector and close-set detectors. With only a tiny memory bank (e.g., 10 images per category) and being training-free, our online learning method could significantly outperform baselines in adapting a detector to novel domains.