Hyneter: Hybrid Network Transformer for Object Detection

TL;DR

Hyneter is a hybrid vision Transformer that combines local and global features to improve object detection, especially for small objects, by integrating CNN and Transformer components.

Contribution

It introduces a Hybrid Network Transformer with a novel backbone and dual switching module to better fuse local and global information in object detection.

Findings

Improved detection accuracy for small objects.

Effective integration of CNN and Transformer features.

Enhanced balance between local and global information.

Abstract

In this paper, we point out that the essential differences between CNN-based and Transformer-based detectors, which cause the worse performance of small objects in Transformer-based methods, are the gap between local information and global dependencies in feature extraction and propagation. To address these differences, we propose a new vision Transformer, called Hybrid Network Transformer (Hyneter), after pre-experiments that indicate the gap causes CNN-based and Transformer-based methods to increase size-different objects result unevenly. Different from the divide and conquer strategy in previous methods, Hyneters consist of Hybrid Network Backbone (HNB) and Dual Switching module (DS), which integrate local information and global dependencies, and transfer them simultaneously. Based on the balance strategy, HNB extends the range of local information by embedding convolution layers into Transformer blocks, and DS adjusts excessive reliance on global dependencies outside the patch.