Light Field Raindrop Removal via 4D Re-sampling

TL;DR

This paper introduces a novel light field raindrop removal network that leverages 4D re-sampling and a new dataset to effectively restore occluded backgrounds and maintain view consistency.

Contribution

The paper proposes a new LFRR network utilizing 4D re-sampling and introduces the first real scene dataset for training and validation.

Findings

Effective raindrop removal and background restoration

Achieves state-of-the-art performance in view consistency

Demonstrates robustness on real scene data

Abstract

The Light Field Raindrop Removal (LFRR) aims to restore the background areas obscured by raindrops in the Light Field (LF). Compared with single image, the LF provides more abundant information by regularly and densely sampling the scene. Since raindrops have larger disparities than the background in the LF, the majority of texture details occluded by raindrops are visible in other views. In this paper, we propose a novel LFRR network by directly utilizing the complementary pixel information of raindrop-free areas in the input raindrop LF, which consists of the re-sampling module and the refinement module. Specifically, the re-sampling module generates a new LF which is less polluted by raindrops through re-sampling position predictions and the proposed 4D interpolation. The refinement module improves the restoration of the completely occluded background areas and corrects the pixel error caused by 4D interpolation. Furthermore, we carefully build the first real scene LFRR dataset for model training and validation. Experiments demonstrate that the proposed method can effectively remove raindrops and achieves state-of-the-art performance in both background restoration and view consistency maintenance.