Osmosis: RGBD Diffusion Prior for Underwater Image Restoration

TL;DR

This paper introduces a novel RGBD diffusion prior trained on outdoor scenes to effectively restore underwater images by removing water effects, despite not being trained on underwater data.

Contribution

The authors propose a diffusion prior that incorporates color and depth information, guided by an underwater image formation model, to improve underwater image restoration.

Findings

Outperforms state-of-the-art methods on challenging underwater scenes

Uses a diffusion prior trained on outdoor RGBD data without underwater images

Generates high-quality clean images from degraded underwater photos

Abstract

Underwater image restoration is a challenging task because of water effects that increase dramatically with distance. This is worsened by lack of ground truth data of clean scenes without water. Diffusion priors have emerged as strong image restoration priors. However, they are often trained with a dataset of the desired restored output, which is not available in our case. We also observe that using only color data is insufficient, and therefore augment the prior with a depth channel. We train an unconditional diffusion model prior on the joint space of color and depth, using standard RGBD datasets of natural outdoor scenes in air. Using this prior together with a novel guidance method based on the underwater image formation model, we generate posterior samples of clean images, removing the water effects. Even though our prior did not see any underwater images during training, our method outperforms state-of-the-art baselines for image restoration on very challenging scenes. Our code, models and data are available on the project website.