ESOD: Efficient Small Object Detection on High-Resolution Images

TL;DR

ESOD introduces a novel framework that efficiently detects small objects in high-resolution images by reusing the detector's backbone for feature-level object seeking and patch-slicing, significantly reducing computation and memory costs.

Contribution

The paper proposes a generic, backbone-reusing framework with a sparse detection head for small object detection on high-res images, outperforming state-of-the-art methods.

Findings

Surpasses SOTA by up to 8% AP on multiple datasets

Reduces computation and GPU memory usage significantly

Effective for CNN- and ViT-based detectors

Abstract

Enlarging input images is a straightforward and effective approach to promote small object detection. However, simple image enlargement is significantly expensive on both computations and GPU memory. In fact, small objects are usually sparsely distributed and locally clustered. Therefore, massive feature extraction computations are wasted on the non-target background area of images. Recent works have tried to pick out target-containing regions using an extra network and perform conventional object detection, but the newly introduced computation limits their final performance. In this paper, we propose to reuse the detector's backbone to conduct feature-level object-seeking and patch-slicing, which can avoid redundant feature extraction and reduce the computation cost. Incorporating a sparse detection head, we are able to detect small objects on high-resolution inputs (e.g., 1080P or larger) for superior performance. The resulting Efficient Small Object Detection (ESOD) approach is a generic framework, which can be applied to both CNN- and ViT-based detectors to save the computation and GPU memory costs. Extensive experiments demonstrate the efficacy and efficiency of our method. In particular, our method consistently surpasses the SOTA detectors by a large margin (e.g., 8% gains on AP) on the representative VisDrone, UAVDT, and TinyPerson datasets. Code is available at https://github.com/alibaba/esod.