WiSE-OD: Benchmarking Robustness in Infrared Object Detection

TL;DR

This paper introduces WiSE-OD, a weight-space ensembling method that enhances robustness of infrared object detection models against distribution shifts and corruptions without extra training or inference costs.

Contribution

The paper proposes WiSE-OD, a novel ensembling technique combining RGB and IR models, and introduces new IR OOD benchmarks, LLVIP-C and FLIR-C, for robustness evaluation.

Findings

WiSE-OD improves robustness across modalities and corruptions.

The method enhances performance without additional training or inference costs.

Benchmark datasets effectively evaluate robustness in IR object detection.

Abstract

Object detection (OD) in infrared (IR) imagery is critical for low-light and nighttime applications. However, the scarcity of large-scale IR datasets forces models to rely on weights pre-trained on RGB images. While fine-tuning on IR improves accuracy, it often compromises robustness under distribution shifts due to the inherent modality gap between RGB and IR. To address this, we introduce LLVIP-C and FLIR-C, two cross-modality out-of-distribution (OOD) benchmarks built by applying corruptions to standard IR datasets. Additionally, to fully leverage the complementary knowledge from RGB and infrared-trained models, we propose WiSE-OD, a weight-space ensembling method with two variants: WiSE-OD$_{ZS}$, which combines RGB zero-shot and IR fine-tuned weights, and WiSE-OD$_{LP}$, which blends zero-shot and linear probing. Evaluated using four RGB-pretrained detectors and two robust baselines on our benchmark and in the real-world out-of-distribution M3FD dataset, our WiSE-OD improves robustness across modalities and to corruption in synthetic and real-world distribution shifts without any additional training or inference costs. Our code is available at: https://github.com/heitorrapela/wiseod.