Physics-Driven Turbulence Image Restoration with Stochastic Refinement

TL;DR

This paper introduces PiRN and PiRN-SR, physics-based neural networks that improve turbulence image restoration by integrating stochastic simulation, leading to better generalization and perceptual quality in real-world conditions.

Contribution

The paper presents a novel physics-integrated neural network framework with stochastic refinement for turbulence image restoration, enhancing generalization and perceptual quality.

Findings

PiRN and PiRN-SR outperform existing methods in real-world turbulence restoration.

The physics-based training improves generalization to unknown turbulence conditions.

Stochastic refinement boosts perceptual quality of restored images.

Abstract

Image distortion by atmospheric turbulence is a stochastic degradation, which is a critical problem in long-range optical imaging systems. A number of research has been conducted during the past decades, including model-based and emerging deep-learning solutions with the help of synthetic data. Although fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions recently, the training of such models only relies on the synthetic data and ground truth pairs. This paper proposes the Physics-integrated Restoration Network (PiRN) to bring the physics-based simulator directly into the training process to help the network to disentangle the stochasticity from the degradation and the underlying image. Furthermore, to overcome the ``average effect" introduced by deterministic models and the domain gap between the synthetic and real-world degradation, we further introduce PiRN with Stochastic Refinement (PiRN-SR) to boost its perceptual quality. Overall, our PiRN and PiRN-SR improve the generalization to real-world unknown turbulence conditions and provide a state-of-the-art restoration in both pixel-wise accuracy and perceptual quality. Our codes are available at \url{https://github.com/VITA-Group/PiRN}.