Task Specific Attention is one more thing you need for object detection

TL;DR

This paper introduces a novel attention-based model called Task Specific Split Transformer (TSST) that achieves state-of-the-art object detection performance on COCO without relying on traditional hand-designed components like anchors and NMS.

Contribution

The paper proposes TSST, a new attention module that splits general-purpose attention into goal-specific parts, enabling simpler and more effective end-to-end object detection models.

Findings

TSST achieves state-of-the-art results on COCO.

The approach eliminates the need for anchors and NMS.

Extensive experiments validate the effectiveness of the method.

Abstract

Various models have been proposed to perform object detection. However, most require many handdesigned components such as anchors and non-maximum-suppression(NMS) to demonstrate good performance. To mitigate these issues, Transformer-based DETR and its variant, Deformable DETR, were suggested. These have solved much of the complex issue in designing a head for object detection models; however, doubts about performance still exist when considering Transformer-based models as state-of-the-art methods in object detection for other models depending on anchors and NMS revealed better results. Furthermore, it has been unclear whether it would be possible to build an end-to-end pipeline in combination only with attention modules, because the DETR-adapted Transformer method used a convolutional neural network (CNN) for the backbone body. In this study, we propose that combining several attention modules with our new Task Specific Split Transformer (TSST) is a powerful method to produce the state-of-the art performance on COCO results without traditionally hand-designed components. By splitting the general-purpose attention module into two separated goal-specific attention modules, the proposed method allows for the design of simpler object detection models. Extensive experiments on the COCO benchmark demonstrate the effectiveness of our approach. Code is available at https://github.com/navervision/tsst