Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model

TL;DR

This paper introduces a physics-inspired transformer model for atmospheric turbulence image restoration and provides two new real-world turbulence datasets for evaluation, addressing the limitations of existing CNN-based methods.

Contribution

A novel transformer-based model designed for turbulence mitigation and the creation of two real-world datasets for comprehensive evaluation.

Findings

Transformer model effectively extracts turbulence distortion maps

New datasets enable evaluation with classical and task-driven metrics

Code and datasets will be publicly available

Abstract

Image restoration algorithms for atmospheric turbulence are known to be much more challenging to design than traditional ones such as blur or noise because the distortion caused by the turbulence is an entanglement of spatially varying blur, geometric distortion, and sensor noise. Existing CNN-based restoration methods built upon convolutional kernels with static weights are insufficient to handle the spatially dynamical atmospheric turbulence effect. To address this problem, in this paper, we propose a physics-inspired transformer model for imaging through atmospheric turbulence. The proposed network utilizes the power of transformer blocks to jointly extract a dynamical turbulence distortion map and restore a turbulence-free image. In addition, recognizing the lack of a comprehensive dataset, we collect and present two new real-world turbulence datasets that allow for evaluation with both classical objective metrics (e.g., PSNR and SSIM) and a new task-driven metric using text recognition accuracy. Both real testing sets and all related code will be made publicly available.