QKVA grid: Attention in Image Perspective and Stacked DETR

TL;DR

This paper introduces SDETR, an improved version of DETR with a novel QKVA grid attention mechanism and stacked architecture, resulting in better performance, especially on small objects, while simplifying training.

Contribution

The paper proposes the QKVA grid for a new perspective on attention and introduces a stacked architecture to enhance DETR's performance and training efficiency.

Findings

SDETR achieves +0.6 AP improvement over DETR.

SDETR outperforms Faster R-CNN on small objects.

The QKVA grid clarifies attention mechanisms in image tasks.

Abstract

We present a new model named Stacked-DETR(SDETR), which inherits the main ideas in canonical DETR. We improve DETR in two directions: simplifying the cost of training and introducing the stacked architecture to enhance the performance. To the former, we focus on the inside of the Attention block and propose the QKVA grid, a new perspective to describe the process of attention. By this, we can step further on how Attention works for image problems and the effect of multi-head. These two ideas contribute the design of single-head encoder-layer. To the latter, SDETR reaches better performance(+0.6AP, +2.7APs) to DETR. Especially to the performance on small objects, SDETR achieves better results to the optimized Faster R-CNN baseline, which was a shortcoming in DETR. Our changes are based on the code of DETR. Training code and pretrained models are available at https://github.com/shengwenyuan/sdetr.