Time-rEversed diffusioN tEnsor Transformer: A new TENET of Few-Shot Object Detection

TL;DR

This paper introduces TENET, a novel tensor transformer approach for few-shot object detection that captures detailed multi-way features and dynamically models query-support correlations, significantly improving detection accuracy.

Contribution

The paper proposes TENET, a tensor transformer with high-order representations and a Transformer Relation Head, addressing information loss and positional sensitivity in existing FSOD methods.

Findings

Achieves state-of-the-art results on PASCAL VOC, FSOD, and COCO datasets.

Effectively captures multi-way feature interactions for robust detection.

Improves sensitivity to positional variations of objects.

Abstract

In this paper, we tackle the challenging problem of Few-shot Object Detection. Existing FSOD pipelines (i) use average-pooled representations that result in information loss; and/or (ii) discard position information that can help detect object instances. Consequently, such pipelines are sensitive to large intra-class appearance and geometric variations between support and query images. To address these drawbacks, we propose a Time-rEversed diffusioN tEnsor Transformer (TENET), which i) forms high-order tensor representations that capture multi-way feature occurrences that are highly discriminative, and ii) uses a transformer that dynamically extracts correlations between the query image and the entire support set, instead of a single average-pooled support embedding. We also propose a Transformer Relation Head (TRH), equipped with higher-order representations, which encodes correlations between query regions and the entire support set, while being sensitive to the positional variability of object instances. Our model achieves state-of-the-art results on PASCAL VOC, FSOD, and COCO.