Multispectral Object Detection with Deep Learning

TL;DR

This paper explores multispectral object detection using thermal and NIR images, training a YOLO v3 model from scratch with data augmentation to improve detection in low visibility conditions.

Contribution

It introduces a novel dataset of thermal and NIR images for object detection and demonstrates training YOLO v3 from scratch on this data.

Findings

Successful detection using multispectral data

Effective data augmentation to prevent overfitting

Enhanced detection in low visibility conditions

Abstract

Object detection in natural scenes can be a challenging task. In many real-life situations, the visible spectrum is not suitable for traditional computer vision tasks. Moving outside the visible spectrum range, such as the thermal spectrum or the near-infrared (NIR) images, is much more beneficial in low visibility conditions, NIR images are very helpful for understanding the object's material quality. In this work, we have taken images with both the Thermal and NIR spectrum for the object detection task. As multi-spectral data with both Thermal and NIR is not available for the detection task, we needed to collect data ourselves. Data collection is a time-consuming process, and we faced many obstacles that we had to overcome. We train the YOLO v3 network from scratch to detect an object from multi-spectral images. Also, to avoid overfitting, we have done data augmentation and tune hyperparameters.