Stereo Waterdrop Removal with Row-wise Dilated Attention

TL;DR

This paper introduces a stereo image-based waterdrop removal method using a novel row-wise dilated attention module and an attention consistency loss, demonstrating superior performance on a newly collected real-world dataset.

Contribution

The work presents a novel stereo waterdrop removal model with a row-wise dilated attention module and attention consistency loss, addressing limitations of single-image approaches.

Findings

Outperforms state-of-the-art methods quantitatively

Achieves better qualitative waterdrop removal results

Introduces a new real-world stereo waterdrop dataset

Abstract

Existing vision systems for autonomous driving or robots are sensitive to waterdrops adhered to windows or camera lenses. Most recent waterdrop removal approaches take a single image as input and often fail to recover the missing content behind waterdrops faithfully. Thus, we propose a learning-based model for waterdrop removal with stereo images. To better detect and remove waterdrops from stereo images, we propose a novel row-wise dilated attention module to enlarge attention's receptive field for effective information propagation between the two stereo images. In addition, we propose an attention consistency loss between the ground-truth disparity map and attention scores to enhance the left-right consistency in stereo images. Because of related datasets' unavailability, we collect a real-world dataset that contains stereo images with and without waterdrops. Extensive experiments on our dataset suggest that our model outperforms state-of-the-art methods both quantitatively and qualitatively. Our source code and the stereo waterdrop dataset are available at \href{https://github.com/VivianSZF/Stereo-Waterdrop-Removal}{https://github.com/VivianSZF/Stereo-Waterdrop-Removal}