Context Enhanced Transformer for Single Image Object Detection

TL;DR

This paper introduces CETR, a novel single image object detection method that incorporates temporal context through a class-wise memory module and test-time adaptation, improving detection efficiency and accuracy.

Contribution

The paper proposes CETR, a new approach that integrates temporal context into single image detection using a memory module and test-time adaptation, reducing complexity of existing methods.

Findings

Effective incorporation of temporal context improves detection accuracy.

Memory module efficiently stores and utilizes temporal information.

Test-time adaptation enhances performance on video datasets.

Abstract

With the increasing importance of video data in real-world applications, there is a rising need for efficient object detection methods that utilize temporal information. While existing video object detection (VOD) techniques employ various strategies to address this challenge, they typically depend on locally adjacent frames or randomly sampled images within a clip. Although recent Transformer-based VOD methods have shown promising results, their reliance on multiple inputs and additional network complexity to incorporate temporal information limits their practical applicability. In this paper, we propose a novel approach to single image object detection, called Context Enhanced TRansformer (CETR), by incorporating temporal context into DETR using a newly designed memory module. To efficiently store temporal information, we construct a class-wise memory that collects contextual information across data. Additionally, we present a classification-based sampling technique to selectively utilize the relevant memory for the current image. In the testing, We introduce a test-time memory adaptation method that updates individual memory functions by considering the test distribution. Experiments with CityCam and ImageNet VID datasets exhibit the efficiency of the framework on various video systems. The project page and code will be made available at: https://ku-cvlab.github.io/CETR.