DEYOLO: Dual-Feature-Enhancement YOLO for Cross-Modality Object Detection

TL;DR

DEYOLO is a novel cross-modality object detection network that effectively fuses RGB and infrared images by mutual enhancement modules, significantly improving detection in poor-illumination environments.

Contribution

The paper introduces DEYOLO, a dual-feature-enhancement network with novel modules for mutual feature enhancement and a bi-directional focus mechanism, advancing cross-modality object detection.

Findings

Outperforms state-of-the-art methods on M3FD and LLVIP datasets.

Effectively reduces interference between RGB and infrared modalities.

Enhances feature representation for better detection accuracy.

Abstract

Object detection in poor-illumination environments is a challenging task as objects are usually not clearly visible in RGB images. As infrared images provide additional clear edge information that complements RGB images, fusing RGB and infrared images has potential to enhance the detection ability in poor-illumination environments. However, existing works involving both visible and infrared images only focus on image fusion, instead of object detection. Moreover, they directly fuse the two kinds of image modalities, which ignores the mutual interference between them. To fuse the two modalities to maximize the advantages of cross-modality, we design a dual-enhancement-based cross-modality object detection network DEYOLO, in which semantic-spatial cross modality and novel bi-directional decoupled focus modules are designed to achieve the detection-centered mutual enhancement of RGB-infrared (RGB-IR). Specifically, a dual semantic enhancing channel weight assignment module (DECA) and a dual spatial enhancing pixel weight assignment module (DEPA) are firstly proposed to aggregate cross-modality information in the feature space to improve the feature representation ability, such that feature fusion can aim at the object detection task. Meanwhile, a dual-enhancement mechanism, including enhancements for two-modality fusion and single modality, is designed in both DECAand DEPAto reduce interference between the two kinds of image modalities. Then, a novel bi-directional decoupled focus is developed to enlarge the receptive field of the backbone network in different directions, which improves the representation quality of DEYOLO. Extensive experiments on M3FD and LLVIP show that our approach outperforms SOTA object detection algorithms by a clear margin. Our code is available at https://github.com/chips96/DEYOLO.