Physical prior guided cooperative learning framework for joint turbulence degradation estimation and infrared video restoration

TL;DR

This paper proposes a novel framework that jointly estimates atmospheric turbulence and restores infrared videos by leveraging physical priors and cooperative learning, significantly improving accuracy in both tasks.

Contribution

It introduces a physical prior guided cooperative learning framework with a cyclic model collaboration and new loss functions, advancing infrared turbulence estimation and image restoration.

Findings

Improved turbulence strength estimation accuracy (lower MAE, higher R2)

Enhanced infrared video restoration quality (higher PSNR)

Validated the effectiveness of physical prior guided cooperative learning

Abstract

Infrared imaging and turbulence strength measurements are in widespread demand in many fields. This paper introduces a Physical Prior Guided Cooperative Learning (P2GCL) framework to jointly enhance atmospheric turbulence strength estimation and infrared image restoration. P2GCL involves a cyclic collaboration between two models, i.e., a TMNet measures turbulence strength and outputs the refractive index structure constant (Cn2) as a physical prior, a TRNet conducts infrared image sequence restoration based on Cn2 and feeds the restored images back to the TMNet to boost the measurement accuracy. A novel Cn2-guided frequency loss function and a physical constraint loss are introduced to align the training process with physical theories. Experiments demonstrate P2GCL achieves the best performance for both turbulence strength estimation (improving Cn2 MAE by 0.0156, enhancing R2 by 0.1065) and image restoration (enhancing PSNR by 0.2775 dB), validating the significant impact of physical prior guided cooperative learning.