DMAT: An End-to-End Framework for Joint Atmospheric Turbulence Mitigation and Object Detection

TL;DR

This paper introduces DMAT, an end-to-end deep learning framework that jointly mitigates atmospheric turbulence effects and enhances object detection accuracy in degraded surveillance imagery.

Contribution

It presents a novel joint learning approach that exchanges features between turbulence mitigation and object detection modules for improved performance.

Findings

DMAT achieves up to 15% better accuracy on turbulence-corrupted datasets.

The framework effectively compensates for spatio-temporal distortions caused by atmospheric turbulence.

Joint training enhances both visualization quality and object detection robustness.

Abstract

Atmospheric Turbulence (AT) degrades the clarity and accuracy of surveillance imagery, posing challenges not only for visualization quality but also for object classification and scene tracking. Deep learning-based methods have been proposed to improve visual quality, but spatio-temporal distortions remain a significant issue. Although deep learning-based object detection performs well under normal conditions, it struggles to operate effectively on sequences distorted by atmospheric turbulence. In this paper, we propose a novel framework that learns to compensate for distorted features while simultaneously improving visualization and object detection. This end-to-end training strategy leverages and exchanges knowledge of low-level distorted features in the AT mitigator with semantic features extracted in the object detector. Specifically, in the AT mitigator a 3D Mamba-based structure is used to handle the spatio-temporal displacements and blurring caused by turbulence. Optimization is achieved through back-propagation in both the AT mitigator and object detector. Our proposed DMAT outperforms state-of-the-art AT mitigation and object detection systems up to a 15% improvement on datasets corrupted by generated turbulence.